Shapely error handling

Fix for https://github.com/opendatacube/datacube-explorer/issues/251
add another layer of exception handling in case .buffer fails.
In this handling, all Multipolygon is broken into Polygons and sequenced in a list,
This list is then filtered for any non-noded error introducing polygon.
No test case added:

unless the entire database is backedup for testing purposes, cannot replicate the error with just a few months of datasets

example log https://github.com/opendatacube/datacube-explorer/issues/251#issuecomment-750727043

root@ip-10-0-13-62:/code# cubedash-gen --force-refresh --refresh-stats --no-init-database s2_l2a
/env/lib/python3.6/site-packages/flask_caching/__init__.py:192: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
  "Flask-Caching: CACHE_TYPE is set to null, "
Updating 1 products for LocalConfig<loaded_from=defaults, environment='default', config={'db_hostname': 'localhost', 'db_port': '5434', 'db_database': 'africa', 'db_username': 'africa', 'db_password': '***', 'index_driver': 'default', 'db_connection_timeout': '60'}>
Generating product summaries...
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
2020-12-24T03:26:25.626335Z [warning  ] summary.footprint.union
Traceback (most recent call last):
  File "/code/cubedash/summary/_model.py", line 95, in add_periods
    shapely.ops.unary_union([p.footprint_geometry for p in with_valid_geometries]) if with_valid_geometries else None
  File "/env/lib/python3.6/site-packages/shapely/ops.py", line 154, in unary_union
    return geom_factory(lgeos.methods['unary_union'](collection))
  File "/env/lib/python3.6/site-packages/shapely/geometry/base.py", line 78, in geom_factory
    raise ValueError("No Shapely geometry can be created from null value")
ValueError: No Shapely geometry can be created from null value
TopologyException: found non-noded intersection between LINESTRING (-1.65696e+07 -6.91138e+06, -1.65696e+07 -6.91138e+06) and LINESTRING (-1.65696e+07 -6.91138e+06, -1.65696e+07 -6.91138e+06) at -16569611.5284 -6911378.0650770711
2020-12-24T03:26:43.032357Z [warning  ] summary.footprint.union.filtering
Traceback (most recent call last):
  File "/code/cubedash/summary/_model.py", line 95, in add_periods
    shapely.ops.unary_union([p.footprint_geometry for p in with_valid_geometries]) if with_valid_geometries else None
  File "/env/lib/python3.6/site-packages/shapely/ops.py", line 154, in unary_union
    return geom_factory(lgeos.methods['unary_union'](collection))
  File "/env/lib/python3.6/site-packages/shapely/geometry/base.py", line 78, in geom_factory
    raise ValueError("No Shapely geometry can be created from null value")
ValueError: No Shapely geometry can be created from null value

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/code/cubedash/summary/_model.py", line 106, in add_periods
    if with_valid_geometries
  File "/env/lib/python3.6/site-packages/shapely/ops.py", line 154, in unary_union
    return geom_factory(lgeos.methods['unary_union'](collection))
  File "/env/lib/python3.6/site-packages/shapely/geometry/base.py", line 78, in geom_factory
    raise ValueError("No Shapely geometry can be created from null value")
ValueError: No Shapely geometry can be created from null value
original list length 95
start index 0
geom length 95
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 94
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 93
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 92
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 91
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 90
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 89
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 88
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 87
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 86
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 85
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 84
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 83
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 82
TopologyException: found non-noded intersection between LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) and LINESTRING (-1.60083e+07 -5.78532e+06, -1.60083e+07 -5.78532e+06) at -16008349.444465393 -5785319.8790948745
start index 81
geom length 81
<class 'list'>
filtered list length 80
s2_l2a done: (1723464 datasets)
done. 1/1 generated, 0 failures
Refreshing statistics...done

One question I have, is whether the shape may be inaccurate now?
It fixes them by dropping polygons, so will Explorer's view of the data/product be potentially misleading in some cases?

One question I have, is whether the shape may be inaccurate now?
It fixes them by dropping polygons, so will Explorer's view of the data/product be potentially misleading in some cases?


Add sortable feature to ArrayField

Prerequisites

[x] I have read the documentation.

Description
In my app, we add many items using array field, and then save it.  We can then edit the saved data later.  Often we run into the problem of wanting to rearrange the order of the array items.  This involves a lot of delete and add which is horribly cumbersome!
I'm looking into making a custom UI widget for the array to allow this capability.  I did this before for a file field, and now I see you have added file field to your project.
So, in the interest of preventing duplicate work, would it be possible to add sortable capabilities to the array field ui widget?  Preferably drag and drop, but up/down arrow moving would work.
Steps to Reproduce

Create an array field
Add a bunch of items
Rearrange those items

Expected behavior
I'd expect the sortable ordering to be represented in a new order in the formData.
Actual behavior
You have to delete item(s) from the array, and then add them again in the order you want.
Version
0.29.1

Duplicate of #217.


Microsoft/AL: Business Central: Button New in page Service Zones cannot be used

We want to create a new Service Zone in the page Service Zones. This page is now empty, but if we choose the button new, then nothing happens.
We have set the option user experience in the company info to Premium. When we then open the screen Service Zones, the this page is empty. Then we choose the button new.
We see that nothing happens.

FWIW: I was curious about this bug and checked on our BC sandbox, and I could add a new Service Zone without problems (notice the extra actions):

I went to the Company Info card, though, and I noticed that the User Experience field was empty (I didn't know this was possible with BC?).
After changing to a Company that has the "Premium" UX, I got the same results as you.

One of the first things I dit in BC was setting the User Experience in the company information on Premium

Sorry, my comment might have been a bit confusing: I was just confirming that I could also reproduce the bug, and that it happened after setting the User Experience setting to "Premium".
Looking at the fields in the page (with the classic development environment), they have "Advanced" set as the "Application Area":
https://docs.microsoft.com/en-us/dynamics365/business-central/across-faq#why-are-some-ui-elements-that-i-used-in-dynamics-nav-not-visible-in-
Advanced is supposed to be assigned to fields that are not commonly used, or that they have not yet been optimised for the web client.
I'm not sure what's the reason for the fields in the zone table to be excluded from BC, but it certainly is very strange to be able to access the page, but then being unable to add any records to it (either the page should be hidden, or the fields should be visible), that's why I think this should be considered a bug.
While waiting to see what Microsoft says regarding this ticket, there's actually a documented way to make this fields available (and everything else marked as Advanced, though):
https://docs.microsoft.com/en-gb/dynamics365/business-central/dev-itpro/developer/devenv-extending-application-areas#adding-advanced-application-area-to-the-essentials-and-premium-experiences-using-an-extension

@salgiza : you are right that you should not be able to have a blank user experience in BC. AFAIK this was due to a bug in 'copy company' where the experience was not copied to the new company. This should be fixed in a next update.
@EelcoWijr : there is inconsistency in the tagging here and that is a bug. I need to confirm with our PM to see whether page 6020 and all links to it should be tagged with #Advanced or #Service. Once confirmed, it will be fixed.
thanks,
Gert

The tags have been updated and will be available in the next update.
br/Gert


Add support for nearley @include

Solves the first part of #26
There is probably still a problem if you include the same file twice from different locations.

The patch looks good! Any chance you can add a sample nearley grammar with the include directive?

Where should that go, into tools/nearley.py, or into the tests dir?

Ideally into tests.

Ok, I have made a unit testcases for the old css grammar test and the include. I added Nearley as a submodule of my own fork as I am currently waiting for an include test to be merged that I made use of.
I will try and add a test for including the same file twice from different files to Nearley itself and then add a test case to Lark. How do you think Lark will handle that case as it stands?

So, it currently fails the new multi-include test with:
  File "lark/load_grammar.py", line 621, in load_grammar
    raise GrammarError("Rule '%s' defined more than once" % name)
GrammarError: Rule 'n_b' defined more than once

Ok, this should be good to go. Not sure if I should do anything about the style errors as I have been following what has been done in existing code.

Thanks, merged!
I made a few small corrections, but overall I left it as you wrote it.


Control SPIR-V debug names during serialization

Right now we emit the MLIR global variable names into the serialized SPIR-V. This bloats the generated SPIR-V blob (and can causes issue for WGSL cause SPIR-V OpName allows more chars than source code symbols). We should plumb through an option across the stack to control the generation of it.

Done via https://reviews.llvm.org/D115531.


Align docs to OSC governance

Changelog
- description: |
    <Update documentation to align with Governance Policy> If needed.

# uncomment types applicable to the change:
  type:

  # - documentation  # change in code docs, haddocks...

Context
https://committees.docs.intersectmbo.org/intersect-open-source-committee/policies/governance/documentation
How to trust this PR
Highlight important bits of the PR that will make the review faster. If there are commands the reviewer can run to observe the new behavior, describe them.
Checklist

[ Y ] Commit sequence broadly makes sense and commits have useful messages
[ ] New tests are added if needed and existing tests are updated. See Running tests for more details
[ Y ] Self-reviewed the diff

Thanks. I've signed the commits and pushed with force.


Specify whether generic is allowed to be infered as {}

I would like to specify that 'To' is not allowed to be inferred as '{}'.
Perhaps some compilation flag, --noInferObjectFromGeneric, or something else.
function convert<To, From>(from: From): To {
    return from as any as To; // doesn't really matter
}

var x = convert(1);  // To = {}

Are you suggesting that the generic be inferred as any?  Essentially {} is equivalent to any anyways.
Default generics allow you to suggest something else to inferred:
function convert<To = string, From = string>(from: From): To {
    return from as any as To; // doesn't really matter
}

var x = convert(1);  // To = string

No, I mean to get a compiler error in such case.

To here is rather meaningless as a type parameter. there is no way for the code at run-time to observe what To is. so this is merely a type cast at compile time. The language already has support for this in the form of type assertions. I would recommend using that instead.

It is just a sample.
I want an option to disallow using such functions.

Can you shed some light on the scenario.

Only if all generic parameters can be deduced either manually or from parameters, the function is allowed to called.
Similar to way it is implemented in C#

i understand. can you share a code sample where you feel you need this?

Let's take this sample:
class Base<T> { field: T; }
class Sub<T> extends Base<T> { constructor(public field: T) { super(); } }
function test<T1, A extends Base<T1>, T2>(arr: A, t2: T2) {
	const data: Array<T1 | T2> = [];
	data.push(arr.field);
	data.push(t2);
	return { input: arr, data };
}

const res = test(new Sub(1), "a"); // T1 = {} , A = Sub<number>, T2 = string
const first: number|string = res.data[0]; // error

Nothing is wrong here and type inference is according to the rules.
The only problem is that T1 inferred as {}/
For my understanding it happens since there is no type specified for T1 from parameters and TSC can choose anything it wants, so {} is good enough.
I want to have such functions to not compile and require writing explicit T1..
I don't want to talk about high-order kinds which solves the problem.
Maybe both samples are different issues.
The main part is that I don't want the implicitly deduced type {} just because there is no other option.

Related to https://github.com/Microsoft/TypeScript/issues/14829

How about this as a workaround (assuming the function requires at least one argument)?
class Base<T> { field: T; }
class Sub<T> extends Base<T> { constructor(public field: T) { super(); } }
const DUMMY = Symbol();
interface T1_was_not_inferred_please_specify_it {
    [DUMMY]: never;
}
function test<T1, A extends Base<T1>, T2>(
    arr: A & ({} extends T1 ? T1_was_not_inferred_please_specify_it : {}), t2: T2) {
    let arr_: A = arr;
    const data: Array<T1 | T2> = [];
    data.push(arr_.field);
    data.push(t2);
    return { input: arr_, data };
}

// error: Argument of type 'Sub<number>' is not assignable to parameter of type 'Sub<number> & T1_was_not_inferred_please_specify_it'.
const res = test(new Sub(1), "a");
const first: number|string = res.data[0];

This is ugly and I wonder if it might have side effects on the way type parameters are inferred, so I support the suggestion.  I have another use case for it that I'll write up if I can think of a simple way to explain it.

Here's another example (thanks @mattmccutchen for the answer there!).

Setting the default to never or unknown is pretty much guaranteed to produce a downstream error. Uninferrable type parameters are a strong anti-pattern regardless; the example code in the OP simply should not be written.

It's not that it doesn't cause an error, it's that the user facing error is not obvious.
This is extremely debuggable:

function X requires 1-2 Generics

This is much less so:

type something something unknown something something doesn't match type something something something something

5k views on this question btw: https://stackoverflow.com/questions/53109837/how-to-make-a-generic-type-argument-required-in-typescript

It is possible to use never to enforce mandatory type parameters.
Consider this function:
function myFunction<T = never, Input extends T = T>(input: Input): T { return input }

This function requires

one parameter
one type parameter
the parameter and type parameter to match

Demonstration:
 // TS-2345: Argument of type 'string' is not assignable to parameter of type 'never'
const value1 = myFunction('hello')

// TS-2345: Argument of type 'number' is not assignable to parameter of type 'string'.(2345)
const value2 = myFunction<string>(123)

// Passes fine!
const value3 = myFunction<string>('hello')

This works because without a T, Input must extend never, which no non-never value can do.


update: Info about NASA API

NASA Public API needs an API Key now.
Thank you for taking the time to work on a Pull Request for this project!
To ensure your PR is dealt with swiftly please check the following:

[x] Your submissions are formatted according to the guidelines in the contributing guide
[x] Your additions are ordered alphabetically
[x] Your submission has a useful description
[x] The description does not end with punctuation
[x] Each table column should be padded with one space on either side
[x] You have searched the repository for any relevant issues or pull requests
[x] Any category you are creating has the minimum requirement of 3 items
[x] All changes have been squashed into a single commit

Sorry for the delay.
So actually some of those APIs still don't need an API key (e.g. TLE API). I will close this PR but feel free to discuss it on https://github.com/public-apis/public-apis/issues/1066.


[stable/postgresql] Fix change ownership of NFS PV (#19921)

What this PR does / why we need it:
This PR fixes issue #19921 as it allows container initialisation even when it cannot change ownership of the mount point itself. A persistent volume is generally mounted in /bitnami/postgresql and a the data folder is created inside. Postgresql does not need owership of the /bitnami/postgresql but only of its contents.
The old code attempted to change ownership of /bitnami/postgresql folder, which fails in certain configurations where a PV of type NFS is mounted. For example, when the folder does not belong to the postgres user but to i.e. nobody:nobody and change ownership is not permitted.
Which issue this PR fixes

fixes #19921

Special notes for your reviewer:
The bug was introduced in https://github.com/helm/charts/pull/17154
This PR should solve the bug and keep compatibility with #17154 needs
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

/assign @sameersbn

Hi, @krufab.
Thanks for your PR! As I said in the related issue (#19921), I'm not really sure about this. The postgresql container does need the permissions in the parent directory (see https://github.com/bitnami/bitnami-docker-postgresql/blob/master/11/debian-9/rootfs/libpostgresql.sh#L739), so with this change the init scripts may not work. But it is true that without this change then the chart isn't working at all.
What do you think about this @juan131 @sameersbn ?

Hi @alemorcuq,
thanks for your answer.
If I read correctly the link you posted in the related issue page (https://github.com/bitnami/bitnami-docker-postgresql/blob/master/11/debian-9/rootfs/libpostgresql.sh#L739), postgres needs WRITE access to the parent folder, but not full ownership. This can be achieved having the shared folder use i.e.  777 mask, so that everyone can write in it.
Probably, full ownership is required for the data folder, but this is different and achieved with the shell intiContainer part.
In general, with Kubernetes and mounted volumes, it's not always possible to change ownership of the parent folder itself, therefore the actual code will fail in many situations.
I increased the minor version because the final effect is different (i.e. if you were relying of a full ownership of the mounted volume, now it't not the case). But I can change it to a patch.

Actually, reading better the actual code, applied with commit 16bea01a62 (the one which changed mindepth from 1 to 0), it does not make much sense. The actual code is:
find {{ .Values.persistence.mountPath }} -mindepth 0 -maxdepth 1 -not -name ".snapshot" -not -name "lost+found" | \
  xargs chown -R {{ .Values.securityContext.runAsUser }}:{{ .Values.securityContext.fsGroup }}

That is, find all files and dirs in /bitnami/postgresql, except ".snapshot" and "lost+found", and change their ownership recursively (-R), but ALSO take /bitnami/postgresql itself (-mindepth 0) and change its ownership recursively...
So it does the job twice and even ignores the "-not -name".
The code previous that commit was:
find {{ .Values.persistence.mountPath }} -mindepth 1 -maxdepth 1 -not -name ".snapshot" -not -name "lost+found" | \
  xargs chown -R {{ .Values.securityContext.runAsUser }}:{{ .Values.securityContext.fsGroup }}

and apparently it was not able to touch .initialized*files.
However, I think it was a matter of permissions, rather than ownership.
So we could either revert to the original -mindepth 1 version or use mine, but the actual code does not make much sense.
Another side note is that yes, my version completes successfully even if it cannot change ownership of some file, however, it chown still prints out an error message which is visible in the container's log

hi @krufab ,
Sorry for the late response, I agree with you in that using -mindepth 0 it will ignore the -not options.
I think we should revert it back to -mindepth 1, so the rules are properly applied. And, as this will not change the permissions of the bitnami/postgresql itself, we should change them but not recursively, so we maintain the ones given the find command.
Do you agree with those changes?

Hi @miguelaeh,
I reverted to -mindepth 1. However, I didn't get the 2nd part of your suggestion.
Should i use chown WITHOUT -R?

Hi @krufab ,
I think so, as we want to give access to the folder but without modify the permissions of the content (those are changed with the find command)

Ok, someone already updated the code, so maybe my PR is obsolete now.
The only thing i might disagree on is that the initContainer fails if xargs chmod cannot change ownership of one item.
Using find ... -exec chmod .. {}; rather than piping to xargs, would allow the script to continue.

Solved by #20479


Fix Android loading %, add early fbig api init, create shortcut, subscribe bot

For #26

Adds #16 #17 but needs to be tested on an actual Android device - uploading isn't working on the Facebook site so can't test at the moment.

Added can_subscribe_bot, subscribe_bot, but have not tested on actual device
Also not sure if this is the correct way to do it, I tried two ways and it didn't seem to make a difference
Also currently the bot is auto subscribed for all apps so this may not even be needed


fix: re-order DB models

Description
Currently broken because Angent*Action models now  depend on agent messages.
Risk

Deploy Plan

LGTM thanks.


Add Deno hosting documentation

Summary
Adds documentation for .NET Aspire Community Toolkit Deno package.
Fixes #1889

Internal previews

📄 File
🔗 Preview link

docs/community-toolkit/hosting-deno.md
.NET Aspire Community Toolkit Deno hosting integration

docs/toc.yml
docs/toc

The other thing that I noticed is that this is missing from the intergrations-overview.md page, there's a section that shows the Available integrations and part of that is a subsection with the Community toolkit integrations.


pythonPackages.qscintilla: enable for python3

Motivation for this change
I plan to add Qgis3 into nixpkgs, and for that qscintilla for python3 is required.
This work is somehow based on https://github.com/NixOS/nixpkgs/pull/38022
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

I plan to add Qgis3 into nixpkgs

Cool! I'm looking forward to having it available on NixOS. #35669
Can we help in any way?


Not getting prediction correctly using the model trained on the custom dataset (similar format as CORD-V2 dataset)

I have trained the Donut model using custom dataset which is on the same line as CORD-v2 dataset. The image is having multiple values in one line and we have around 23 to 24 lines in each document. I have used the base model as "naver-clova-ix/donut-base".
I am using 149 documents for the training and following is the breakup of the datasets
training = 119 images
validation = 22 images
testing = 8 images
I have crated 3 meradata.jsonl file i.e. for train, validation and test. Below is the sample value from the metadat.jsonl file from the training database
{"file_name": "IOB_Bank_31_image_0.jpg", "ground_truth": "{\"gt_parse\": {\"bank_stmt_entries\": [{\"TXN_DATE\": \"02-11-2023\", \"TXN_DESC\": \"SB Int: 10-2023:0\", \"CHEQUE_REF_NO\": null, \"WITHDRAWAL_AMT\": null, \"DEPOSIT_AMT\": \"93.00\", \"BALANCE_AMT\": \"10901.92\"}, {\"TXN_DATE\": \"09-12-2023\", \"TXN_DESC\": \"CHRGS- SMS ALERT\", \"CHEQUE_REF_NO\": null, \"WITHDRAWAL_AMT\": \"1.06\", \"DEPOSIT_AMT\": null, \"BALANCE_AMT\": \"10900.86\"}, {\"TXN_DATE\": \"02-02-2024\", \"TXN_DESC\": \"Debit Card AMC-2\", \"CHEQUE_REF_NO\": null, \"WITHDRAWAL_AMT\": \"295.00\", \"DEPOSIT_AMT\": null, \"BALANCE_AMT\": \"10605.86\"}, {\"TXN_DATE\": \"02-02-2024\", \"TXN_DESC\": \"SB Int: 01-2024: 0\", \"CHEQUE_REF_NO\": null, \"WITHDRAWAL_AMT\": null, \"DEPOSIT_AMT\": \"75,00\", \"BALANCE_AMT\": \"10680.86\"}]}}"}
I trained the model for 30 epochs and following are the values for loss and val_edit_distance
loss = 0.03544
val_edit_distance = 0.3443
Following is the config parameters used for the training

"max_epochs":30,
"val_check_interval":0.2, # how many times we want to validate during an epoch
"check_val_every_n_epoch":1,
"gradient_clip_val":1.0,
"num_training_samples_per_epoch": 119,
"lr":3e-5,
"train_batch_sizes": [2],
"val_batch_sizes": [1],
"num_nodes": 1,
"warmup_steps": 180, # 800/8*30/10, 10%
"result_path": "/content/drive/MyDrive/universal-bank-statement-reader/processed-dataset/result",
"verbose": True,

When I am trying to find the prediction using the test dataset, I am getting following output because I had put the print statement at specific location
seq ==>: <s_bank-stmt>署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署12-323310-3012-510-3021-32-2021-2021-2021-2021-2021-2021-2021-2021-3021-32419181mt-3021-3241.4351.43<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.43.43.
seq after token2json ==>: {'text_sequence': '署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署署12-323310-3012-510-3021-32-2021-2021-2021-2021-2021-2021-2021-2021-3021-32419181mt-3021-3241.4351.43<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.43.43.'}
ground_truth after json load ==>: {'gt_parse': {'bank_stmt_entries': [{'TXN_DATE': '02-11-2023', 'TXN_DESC': 'SB Int: 10-2023:0', 'CHEQUE_REF_NO': None, 'WITHDRAWAL_AMT': None, 'DEPOSIT_AMT': '93.00', 'BALANCE_AMT': '10901.92'}, {'TXN_DATE': '09-12-2023', 'TXN_DESC': 'CHRGS- SMS ALERT', 'CHEQUE_REF_NO': None, 'WITHDRAWAL_AMT': '1.06', 'DEPOSIT_AMT': None, 'BALANCE_AMT': '10900.86'}, {'TXN_DATE': '02-02-2024', 'TXN_DESC': 'Debit Card AMC-2', 'CHEQUE_REF_NO': None, 'WITHDRAWAL_AMT': '295.00', 'DEPOSIT_AMT': None, 'BALANCE_AMT': '10605.86'}, {'TXN_DATE': '02-02-2024', 'TXN_DESC': 'SB Int: 01-2024: 0', 'CHEQUE_REF_NO': None, 'WITHDRAWAL_AMT': None, 'DEPOSIT_AMT': '75,00', 'BALANCE_AMT': '10680.86'}]}}
ground_truth ==>: {'bank_stmt_entries': [{'TXN_DATE': '02-11-2023', 'TXN_DESC': 'SB Int: 10-2023:0', 'CHEQUE_REF_NO': None, 'WITHDRAWAL_AMT': None, 'DEPOSIT_AMT': '93.00', 'BALANCE_AMT': '10901.92'}, {'TXN_DATE': '09-12-2023', 'TXN_DESC': 'CHRGS- SMS ALERT', 'CHEQUE_REF_NO': None, 'WITHDRAWAL_AMT': '1.06', 'DEPOSIT_AMT': None, 'BALANCE_AMT': '10900.86'}, {'TXN_DATE': '02-02-2024', 'TXN_DESC': 'Debit Card AMC-2', 'CHEQUE_REF_NO': None, 'WITHDRAWAL_AMT': '295.00', 'DEPOSIT_AMT': None, 'BALANCE_AMT': '10605.86'}, {'TXN_DATE': '02-02-2024', 'TXN_DESC': 'SB Int: 01-2024: 0', 'CHEQUE_REF_NO': None, 'WITHDRAWAL_AMT': None, 'DEPOSIT_AMT': '75,00', 'BALANCE_AMT': '10680.86'}]}
evaluator ==>: <donut.util.JSONParseEvaluator object at 0x7d697edbfc10>
score ==>: 0
I had referred following URL as reference
https://github.com/NielsRogge/Transformers-Tutorials/blob/master/Donut/CORD/Fine_tune_Donut_on_a_custom_dataset_(CORD)_with_PyTorch_Lightning.ipynb
Please help me out in identifying and revolve the issue and let me know if you need more information
Thank you in advance

@SiriusPoint any updates? I have the same problem

@CarlosSerrano88, Not yet. I am trying but not getting appropriate results.

The transformers implementation of Donut seems to have broken saving and loading at some point. Try transformers==4.26.1 and see if that works.

any updates? I have the same problem

+1

with transformers==4.25.1 working perfect!

I have the same problem with the last version of transformers. Going back to 4.40.1, and the saved model works again

与 transformers==4.25.1 完美协作！

Did you solve the similar question when training on cord-v2 by using transformers 4.25.1?


Import calendar error

When using traccar with postgres I get this error when I try to import an ics file.
  ERROR: column "data" is of type oid but expression is of type bytea
  Hint: You will need to rewrite or cast the expression.
  Position: 68 - PSQLException (... < QueryBuilder:475 < DataManager:508 < CalendarManager:100 < ...)

I have tried files generated from iCalendar, Google Calendar and Thunderbird.

@Abyss777, have you tested calendar blob with postgresql? Can it be an old version on the database?

I have tested it on PostgreSQL 9.4 and 9.6.
The only solution I found was changing the column typecalendars.data from oid to bytea after Traccar database creation.

I remember that I have tested, looks like behavior was changed some time ago:
https://liquibase.jira.com/browse/CORE-1863
Possible solution:
https://stackoverflow.com/questions/42388886/force-liquibase-to-map-blob-to-bytea-on-posgresql

My theoretical solution is:
  <changeSet author="author" id="changelog-3.12-pgsql">

    <preConditions onFail="MARK_RAN">
      <dbms type="postgresql" />
      <sqlCheck expectedResult="oid">SELECT data_type FROM information_schema.columns WHERE table_name = 'calendars' AND column_name = 'data';</sqlCheck>
    </preConditions>
    <dropColumn tableName="calendars" columnName="data" />
    <addColumn tableName="calendars">
      <column name="data" type="bytea">
        <constraints nullable="false" />
      </column>
    </addColumn>

  </changeSet>

I'm assuming that if it is oid then it is impossible to create calendar and table is empty. We can easily drop column and recreate with right type

Works OK
INFO 6/6/17 1:30 PM: liquibase: ./schema/changelog-master.xml: changelog-3.12::changelog-3.12-pgsql::author: Column calendars.data dropped
INFO 6/6/17 1:30 PM: liquibase: ./schema/changelog-master.xml: changelog-3.12::changelog-3.12-pgsql::author: Columns data(bytea) added to calendars
INFO 6/6/17 1:30 PM: liquibase: ./schema/changelog-master.xml: changelog-3.12::changelog-3.12-pgsql::author: ChangeSet changelog-3.12::changelog-3.12-pgsql::author ran successfully in 95ms

Can you please send a pull request.


Add badge for android-gems.com

Added badge for android-gems: http://www.android-gems.com/lib/yshrsmz/KeyboardVisibilityEvent , it looks like this:

thanks!
but could you place the badge below the badge of Android Arsenal?
I want to place badges in order, in one place.


Model dir filepath should be saved to metadata.yml

The most efficient solution to convert class IDs into strings might be to do it inside the PoseReader where the JSON metadata file for the model is already accessed to retrieve part labels, etc.


HEVC hardware encoder worked well in Gen11 i3, not in Gen10 and Gen11 i7

The same windows 10 app got the difference result in the different kind of computers, why?
The MSDK HEVC hardware encoder worked well:

Sometime works, but often return the -17 error:

Always get the -3 when init the encoder:

Hi @jiangjianping ,
What app are you running? Propriety one or sample_encode ? Please try to reproduce via sample_encode and share cmd line and console output.

@dmitryermilov
My own app which is using media sdk.  I will try to reproduce this via sample_encode. Thanks.

this issue can be closed.


Sent messages shown as "Sent Failed. Retry." when rendering chat using existing conversation

Is it an issue related to Adaptive Cards?

[ ] Yes, this is an Adaptive Card issue but it is specific to Web Chat.

Do you have any screenshots?

What version of Web Chat are you using?
Latest production
Which distribution are you using Web Chat from?
Bundle (webchat.js)
Which hosting environment does this issue primarily affect?
Web apps
Which browsers and platforms do the issue happened?
Browser: Chrome (latest)
Which area does this issue affect?
Chat history
Is this an accessibility issue?

[ ] Yes, this is an accessibility issue.

Please describe the bug

Start a conversation with a bot by making a POST to https://directline.botframework.com/v3/directline/conversations
Use that token to render the webchat on a page:
window.WebChat.renderWebChat(    { directLine: window.WebChat.createDirectLine({ token: token, }),    styleSet: getBotStyleSet(),    sendTypingIndicator: true,    store,    styleOptions:    getBotStyleOptions(),  }, document.getElementById(ctrl.widgetId + "-container") );
Start a conversation with the chatbot, sending a message
Refresh the page using the same token (for the existing conversation)
The chatbot will render the existing conversation, but the messages that were sent by the user, will appear as "Sent Failed. Retry.", see pic above.

Do you see any errors in console log?
No
How to reproduce the issue?
Start a conversation with a bot by making a POST to https://directline.botframework.com/v3/directline/conversations

Use that token to render the webchat on a page:
window.WebChat.renderWebChat(    { directLine: window.WebChat.createDirectLine({ token: token, }),    styleSet: getBotStyleSet(),    sendTypingIndicator: true,    store,    styleOptions:    getBotStyleOptions(),  }, document.getElementById(ctrl.widgetId + "-container") );
Start a conversation with the chatbot, sending a message
Refresh the page using the same token (for the existing conversation)

What is the expected and actual behavior?
Expected:

The chatbot should render the existing conversation, and the messages should appear as "Sent"
Actual:
The chatbot will render the existing conversation, but the messages that were sent by the user, will appear as "Sent Failed. Retry.", see pic above.

Adaptive Card JSON
No response
Additional context
No response

This is duplicate of #4501.


Interpret with-continuation-mark

This will require the implementation of delimited continuations into the interpreter. Something I shall tackle next.


durable-objects: clarify HTTP requests, RPC method calls, etc - pricing

Add the upcoming RPC method calls to the pricing so it is clear how much they cost.

Superseded.


Add hybrid changes

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Please ensure to add changelog with this PR by answering the following questions.

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet. - During March
When you expect to publish swagger? Please provide date, or month to public if date is not available yet. Estimate September.
If it's an update to existing version,  please select SDKs of specific language and CLIs that require refresh after swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[x] No, no need to refresh for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[x] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[ ] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

LGTM

@akning-ms
Please hold on and not merge until @nemijato and I decide the merging schedule. Thanks!

Please go through the contribution checklist added as part of the review and mark them completed. It will help you fix most of the issues and improve the quality of swagger and help close the PR faster. This is a requirement before review can be continued.

@akning-ms
Please hold on and not merge until @nemijato and I decide the merging schedule. Thanks!

Got it, next time, you can a label "DoNotMerge" and remove it when it is ready for merge

@akning-ms
Please hold on and not merge until @nemijato and I decide the merging schedule. Thanks!

Got it, next time, you can a label "DoNotMerge" and remove it when it is ready for merge

oh. I just know this tag 👍
Thanks!

Please go through the contribution checklist added as part of the review and mark them completed. It will help you fix most of the issues and improve the quality of swagger and help close the PR faster. This is a requirement before review can be continued.

@chiragg4u
The track2 validation issue is not introduced by this PR.
I will fix the track2 validation issue when I merge https://github.com/Azure/azure-rest-api-specs/tree/dev-sql-Microsoft.Sql-2020-11-01-preview into the master branch.


[FEATURE] Ghouls and Mortals PCs

There is rules for mortals and ghouls PCs in the Companion, and I think that would be a nice thing to add. You seem to add quite a lot on your plate so I will try to do it and provide you with a PR.
It's mostly refactoring anyway : most rules are common between ghouls, mortals and vampires, it just need some cleanup here and there.

Wrong account.


Set different time for each segment

#39

I just pushed a fix where calling next() method would leave the previous bar incomplete while the next story is paused.
Here is the code snipped that may clarify the issue
  override fun onPage(oldPageIndex: Int, newPageIndex: Int) {
                spb.pause()
                Handler().postDelayed({
                    spb.start()
                }, 2000)
                // New page started animating
            }

here we're simulating downloading a story scenario by adding a delay, and then start the story after the delay.
Here is a video of the issue
https://user-images.githubusercontent.com/8546946/160190551-941ea378-8bc8-4743-ab89-9e58638d41d0.mov


createReducer cannot be called when handling more than 2 actions

Description
Invoking a reducer which was created with more than 2 chained handleAction() calls does not compile.
Steps to Reproduce
Following produces a ts compile error:
import {ActionType, createReducer, createStandardAction} from "typesafe-actions";

export type State = {
    foo: string,
};

const defaultState: State = {
    foo: "",
};

const actions = {
    foo1: createStandardAction("foo1")(),
    foo2: createStandardAction("foo2")(),
    foo3: createStandardAction("foo3")(),
    foo4: createStandardAction("foo4")(),
};

type FooActionsType = ActionType<typeof actions>;

declare module "typesafe-actions" {
    export type RootAction = FooActionsType;
}

const reducer = createReducer(defaultState)
    .handleAction(actions.foo1, (state) => {
        return {
            ...state,
        };
    })
    .handleAction(actions.foo2, (state) => {
        return {
            ...state,
        };
    })
    .handleAction(actions.foo3, (state) => {
        return {
            ...state,
        };
    })
    .handleAction(actions.foo4, (state) => {
        return {
            ...state,
        };
    });

reducer(defaultState, actions.foo1()); // error
reducer(defaultState, actions.foo2()); // error
reducer(defaultState, actions.foo3()); // ok
reducer(defaultState, actions.foo4()); // ok

Running tsc:
reducer.ts:46:23 - error TS2345: Argument of type 'EmptyAction<"foo1">' is not assignable to parameter of type 'EmptyAction<"foo3"> | EmptyAction<"foo4">'.
  Type 'EmptyAction<"foo1">' is not assignable to type 'EmptyAction<"foo3">'.
    Type '"foo1"' is not assignable to type '"foo3"'.

46 reducer(defaultState, actions.foo1()); // error
                         ~~~~~~~~~~~~~~

reducer.ts:47:23 - error TS2345: Argument of type 'EmptyAction<"foo2">' is not assignable to parameter of type 'EmptyAction<"foo3"> | EmptyAction<"foo4">'.
  Type 'EmptyAction<"foo2">' is not assignable to type 'EmptyAction<"foo3">'.
    Type '"foo2"' is not assignable to type '"foo3"'.

47 reducer(defaultState, actions.foo2()); // error
                         ~~~~~~~~~~~~~~

Found 2 errors.

If i remove 2 handleAction calls from the reducer, it compiles.
Project Dependencies
{
  "dependencies": {
    "typesafe-actions": "4.1.1"
  },
  "devDependencies": {
    "typescript": "3.4.4"
  }
}

tsconfig.json:

{
  "compilerOptions": {
    "strict": true,
    "noEmit": true
  },
  "exclude": [
    "node_modules"
  ]
}

Environment (optional)

OS: macOS 10.14.2
Node Version: v11.14.0
Package Manager and Version: yarn v1.15.2

Hey @zaytsevfuu
I love to receive so detailed reports, thanks ;)
I identified a really tiny bug in recursive conditional type algorithm. I added a new test case to my test suite. Will be released in a few minutes.
Best!

Hey @piotrwitek,
thank you for the fast reaction.
It looks like there is still something not working correctly.
Now i am getting this error: Property 'payload' does not exist on type '{}'. when trying to access the action payload.
Also it looks like that the original error i described, was gone with 4.1.2.
I get the payload error with 4.1.2 and 4.1.3
The payload attribute was working with 4.1.1.
import {ActionType, createReducer, createStandardAction} from "typesafe-actions";

export type State = {
    foo: string,
};

const defaultState: State = {
    foo: "",
};

const actions = {
    foo1: createStandardAction("foo1")<string>(),
    foo2: createStandardAction("foo2")<string>(),
    foo3: createStandardAction("foo3")<string>(),
    foo4: createStandardAction("foo4")(),
};

type FooActionsType = ActionType<typeof actions>;

declare module "typesafe-actions" {
    export type RootAction = FooActionsType;
}

const reducer = createReducer(defaultState)
    .handleAction(actions.foo1, (state, action) => {
        const payload = action.payload;

        return {
            ...state,
        };
    })
    .handleAction(actions.foo2, (state, action) => {
        const payload = action.payload;

        return {
            ...state,
        };
    })
    .handleAction(actions.foo3, (state, action) => {
        const payload = action.payload;

        return {
            ...state,
        };
    })
    .handleAction(actions.foo4, (state, action) => {
        return {
            ...state,
        };
    });

reducer(defaultState, actions.foo1(""));
reducer(defaultState, actions.foo2(""));
reducer(defaultState, actions.foo3(""));
reducer(defaultState, actions.foo4());

tsc gives me:
reducer.ts:26:32 - error TS2339: Property 'payload' does not exist on type '{}'.

26         const payload = action.payload;
                                  ~~~~~~~

reducer.ts:33:32 - error TS2339: Property 'payload' does not exist on type '{}'.

33         const payload = action.payload;
                                  ~~~~~~~

reducer.ts:40:32 - error TS2339: Property 'payload' does not exist on type '{}'.

40         const payload = action.payload;
                                  ~~~~~~~

Found 3 errors.

Hey @zaytsevfuu , no problem!
You just need to update the type injection protocol, which was changed to this:
declare module './type-helpers' {
  interface Types {
    RootAction: ActionType<typeof actions>;
  }
}

That did the trick! Thank you!


Feature/assets

📋 Information
Summary
Pushing the right base branch :skull:
Details
Assets
Documented Assets
Using C++20 now!

I think I am just brainrot from the inside


fix logging levels and go micro default log level

Description
We've fixed the configuration of logging levels. Previously it was not possible
to configure a service with a more or less verbose log level then all other services
when running in the supervised / runtime mode ocis server.
For example OCIS_LOG_LEVEL=error PROXY_LOG_LEVEL=debug ocis server did not configure
error logging for all services except the proxy, which should be on debug logging. This is now fixed
and working properly.
Also we fixed the format of go-micro logs to always default to error level.
Previously this was only ensured in the supervised / runtime mode.
Related Issue

Fixes #4089

Motivation and Context

How Has This Been Tested?

test environment:
test case 1:
test case 2:
...

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Technical debt
[ ] Tests only (no source changes)

Checklist:

[ ] Code changes
[ ] Unit tests added
[ ] Acceptance tests added
[ ] Documentation ticket raised:

:boom: Acceptance test cs3ApiTests-ocis failed. Further test are cancelled...

@rhafer I also removed leftovers from the runtime. The setMicroLogger() will be impolitely called with the import of "github.com/owncloud/ocis/v2/ocis-pkg/log"


api-umbrella admin gui reorder url prefixes seems not to work at all

Hello,
Using the api-umbrella admin gui through api.data.gov I would like to use the "reorder" button to restructure my url prefixes into a better organization.  However no amount of fumbling about will persist the reordering.  This includes making sure to press the DONE button and then the SAVE button and even the PUBLISH step.  When I return the url prefixes are back to their original first-in-at-the-top ordering.
Using API Umbrella Version 0.16.0-pre1 with Chrome 73.
Thanks,
Paul

Did you manage to find out something on this topic?

Nope, not really expecting a fix at this point.  Thanks for the reminder as I will just close the ticket.  Does your organization have an alternative api gateway they are moving towards?

Hello @pauldzy, I have occasionally had a need to re-order things in the umbrella gui.
Are you talking about the priority between different api backends, or the urls inside each one?
I have been able to update priority between the backends, but no luck on the urls inside.
For the backend priority, I have the admin apis in a postman and I call Get route with the id from the gui, I remove the "api" level from the object, and then send it as a PUT to update it. This did not work on the urls though, which makes me wonder if the order is even being saved in the DB.


iOS/Android - App returns to account settings instead of workspace list after deleting workspace

Explanation of Change

Originally, after removing the active workspace, the navigateWithSwitchPolicyID method was called, as a result, a new settings screen was pushed without the selected policyID (following the rule that the last bottom tab screen stores the ID of the active policy). To fix this error, instead of pushing a new settings screen, params of the current settings screen are modified.
Fixed Issues

$ https://github.com/Expensify/App/issues/53352
PROPOSAL: https://github.com/Expensify/App/issues/53352#issuecomment-2548741552
Tests

[ ] Verify that no errors appear in the JS console

Launch the app.
Open the workspace switcher and select the workspace that will be deleted in next steps.
Open the settings tab.
Open the workspaces list.
Delete the active workspace and verify if the app remains on the same page.
Go back to the Inbox tab and repeat steps 2-4.
Open the active workspace.
Go to the Profile page.
Delete the workspace and verify if the workspace list page has been opened.

Offline tests

Same as tests
QA Steps

Same as tests

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I used JaimeGPT to get English > Spanish translation. I then posted it in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.ts or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] I added unit tests for any new feature or bug fix in this PR to help automatically prevent regressions in this user flow.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/user-attachments/assets/447f9d84-04be-4d96-b5d7-07c525dbf4ee

Android: mWeb Chrome

https://github.com/user-attachments/assets/7d5c469f-56b2-42aa-8ef8-3460f09d48ce

iOS: Native

https://github.com/user-attachments/assets/db9bfd5c-b676-451b-9b00-f5691c068b2e

iOS: mWeb Safari

MacOS: Chrome / Safari

https://github.com/user-attachments/assets/3ff23bc9-6ba6-4ed6-a61b-3283d4f9cc43

MacOS: Desktop

https://github.com/user-attachments/assets/3964efd3-8b90-4b99-8f45-cf1d6fae6084

@ishpaul777
Thanks for finding this issue 🙌, it has been already resolved as well as lint checks. Now it should work fine. About unit tests, logic fixing this bug will be changed in the PR that should be merged soon. We will try to include tests there for this case


Some questions about schedule_sampling？

Dear @ZhengChang467 ：
Thanks for your excellent codes. I have two questions about the function of schedule_sampling.

what is it used for? 2) why should we use this strategy?

Thanks for your attention to our work. We use schedule_sampling to help stabilize the training process. Schedule_sampling can help dynamically update the percentage of the ground truth images in the input images. For example, at the beginning of the training, most of the input images are the ground truth instead of the predictions.  However, at the end of the training, most input images are predictions. The Schedule_sampling can help models progressively fit the training data and stabilize the training process.


kingarthur's updates for Factorio 0.15

Tweaks to research & entity data from @kingarthur91 (tagged as the commit author). I'm just the committer, filtering out the line-ending changes and such that can be really hard to deal with if you have little Git experience. (Easy solution: Put * text=auto in a .gitattributes file at the repo root, and Git will sort it out for you.) Source files for the updates I committed here came from this forum post by kingarthur.
I took the liberty of holding back changes that appeared related to renaming the mod "Warehousing v15", but I can commit and add that set of changes in just a few keystrokes if @Anoyomouse wants them. I operated under the (possibly incorrect) assumption that these changes will go into an updated version of the mod currently published on the mod portal.
Attempted to do the sensible thing with version numbers, and matched the values in info.json and control.lua. I apologize if that is incorrect.

Hey
Thanks for doing this for me ...
I don't think needing high tech packs are required for the warehouses, they don't ahve anything fancy to them, jsut lots of storage space, so i'd personally prefer cutting that requirement
otherwise i'll pull it into the pack

Did I miss something that needs to be fixed?

Hey, sorry about that, i've been buzy with work, and other things and have not had time to merge your changes.
I've done that now, however i still need to bundle it for a release on the mod portal
Any chance you can send me a message to pull in these changes so i can quickly deploy it to the mod portal?

No worries, was just wondering if I'd fixed up the research wrong or something.
What do you need in the message? (I presume on the Factorio forums.)


Fix Streams Scala foreach recursive call

Due to lack of conversion to kstream Predicate, existing foreach method in KStream.scala would result in StackOverflowError.
This PR fixes the bug and adds testing for it.

@vvcephei could you take a look?

This PR is good to go.

Can you explain the mechanism by which the call becomes recursive? I feel like I'm missing it...

Can you please add a brief description? The description will become the commit message when we squash and merge.
Also, does it make sense to reference KAFKA-7316 also in this PR's title? You could drop a quick comment on that ticket that forEach is also affected.

Basically it's doing an implicit conversion back to the Scala API throw the ImplicitConversions._ import

Oh, I get it. I'd forgotten that we have (e.g.) implicit def wrapKStream[K, V](inner: KStreamJ[K, V]): KStream[K, V]

Are we good with this one @vvcephei ?

I will, in an hour or so.

@joan38 Also, please see my note on privatizing FunctionConversions. I misread it yesterday.

Sure I can do that.

@guozhangwang All good to merge

Retest this please

@guozhangwang This needs to be cherry-picked to 2.0


Laravel 5.8 Shift

This pull request includes the changes for upgrading to Laravel 5.8. Feel free to commit any additional changes to the shift-12998 branch.
Before merging, you should:

Checkout the shift-12998 branch
Review all pull request comments for additional changes
Update your dependencies for Laravel 5.8
Run composer update (if the pre-scripts fail, add --no-scripts)
Thoroughly test your application

If you want help with your upgrade, check out the Shift Human Services or join the Shifty Coders Slack room to get answers to all your Laravel questions.

:x: Shift could not upgrade the following configuration files since they differed from the default Laravel version. You will need to compare these configuration files against the Laravel 5.8 configuration files and merge any changes:

[ ] config/app.php
[ ] config/database.php
[ ] config/logging.php
[ ] config/queue.php
[ ] config/session.php

:x: Laravel 5.8 has a development dependency for mockery/mockery of ^1.0. You have a modified dependency for mockery/mockery in your composer.json. You should review your dependency and ensure it meets the new version requirement.

:warning: Laravel 5.8 introduced a set of AWS environment variables to help consolidate configuring the various AWS services. You should rename the following native environment variables to their new names. While you don't have to rename any custom environment variables you created for AWS services, you should consider following the new naming convention.

SQS_KEY to AWS_ACCESS_KEY_ID
SQS_SECRET to AWS_SECRET_ACCESS_KEY
SQS_REGION to AWS_DEFAULT_REGION
SES_KEY to AWS_ACCESS_KEY_ID
SES_SECRET to AWS_SECRET_ACCESS_KEY
SES_REGION to AWS_DEFAULT_REGION

:x: Laravel 5.8 has a dependency for php of ^7.1.3. You have a modified dependency for php in your composer.json. You should review your dependency and ensure it meets the new version requirement.

:x: Shift detected your application has a development dependency for the laravel/telescope package. The recommended version for Laravel 5.8 is ^2.0. You should review your dependency and ensure it meets the new version requirement, as well as review their README for additional changes.

:information_source: Laravel 5.8 changed the minimum requirement for password length from 6 characters to 8 character. While this change is not required, you should consider this opportunity to review the password policy for your application.

:warning: Laravel 5.8 changed several of the core contracts with new implementations and methods. You should review the Upgrade Guide for more detail on these respective changes:

Illuminate\Contracts\Validation\Validator

Shift found references to these contracts within:

[ ] app/Http/Controllers/Auth/RegisterController.php

:warning: If you are using JSON values in your MySQL or MariaDB tables, the query builder now returns unquoted JSON values in Laravel 5.8.

:warning: If you are using SQLite, the oldest supported version in Laravel 5.8 is SQLite 3.7.11. Laravel recommends running SQLite 3.8.8 or higher.

:warning: The Nexmo and Slack notification channels have been extracted into first-party packages. To use these channels in your application, run the following command to require the respective packages:
composer require laravel/nexmo-notification-channel
composer require laravel/slack-notification-channel

:warning: Laravel 5.8 correctly pluralizes multi-word model names ending in a word with an irregular pluralization. For example, Laravel 5.7 incorrectly pluralized the table name for the UserFeedback model as user_feedbacks. Laravel 5.8 correctly pluralizes this table name as user_feedback.
If you were relying on this irregular pluralization, you should rename your table name or set the $table property for your model. If you were overriding this behavior by setting the $table property, you may remove it and now rely on the framework.

:x: PHP syntax errors were detected after running your Shift. Often these are simply differences between the PHP version of Shift and your project. Occasionally they are misplaced lines.
You can check the PHP syntax by running php -l on the following files:

[ ] app/Object.php

:tada: Congratulations, you're now running the latest version of Laravel. You can now use some of the micro Shifts to ensure your app is fully upgraded and following the Laravel recommended and community practices.

Laravel Linter analyzes your codebase for potential opportunities to do things "The Laravel Way".
Laravel Fixer automates many of the suggestions found by the Analyzer and Linter.
Upgrade Checker ensures your application is fully upgraded by detecting any outdated code.


5.0.0-preview.7 install instructions filename contains an extra space character

Announcements and pages about the 5.0.0-preview.7 release link to this url when linking to the installation instructions for Linux. (eg, on the Download .NET 5.0 page) Note that this points to the file release-notes/5.0/preview/5.0.0-preview.7-install-instructions.md in this repo.
However, this file is actually named release-notes/5.0/preview/5.0.0-preview.7-install-instructions .md (note the extraneous space character before the file extension), which causes all the Linux installation instructions links to be 404s.

Thanks @justinian. This was fixed by PR #4980


TWAI was panicked for ESP32-S3

Bug description

Twai driver crashes with invalid DLC. I'm using ESP32-S3 to receive CAN packets using embassy-twai example, and sometimes I encounter this error.
From my debugging, this issue occurs when TWAI receives a frame with DLC > 8 (which is unexpected because my system uses Basic CAN, where no frame should have DLC > 8). The issue is difficult to reproduce since it randomly occurs after 30 minutes or more. I will continue debugging to identify the root cause.
Do you guys have any ideas with that case ?

To Reproduce

T.B.D

Expected behavior
We need to handle the case where DLC is greater than 8 because it's possible. I checked esp-idf, and they handle the DLC by adding a bit to indicate DLC_NON_COMP in their TWAI frame struct. Therefore, I suggest adding the following:
pub enum EspTwaiError {
    /// TWAI peripheral has entered a bus-off state.
    BusOff,
    /// Data length code is larger than 8. This will break compliance with ISO 11898-1.
    DlcNoneComp,
    /// Encapsulates errors defined by the embedded-hal crate.
    EmbeddedHAL(ErrorKind),
}

Environment

Target device: ESP32-S3
Crate name and version: esp-hal

Hello, thank you for your report. Please include which esp-hal version you're using.

Yes, current latest main branch also occurs. I also tested with esp-hal 0.20.1. I just added version 0.20.1 on description


examples render black (for me)

After installing the -lSDL2 dependency:
apt-get install libsdl2-dev

I can compile and run the examples (on Debian), but they all render black without error messages:
$ uname -a
Linux purism 4.15.0-2-amd64 #1 SMP Debian 4.15.11-1 (2018-03-20) x86_64 GNU/Linux
$ cargo run --example hello_world
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/examples/hello_world`
OpenGL Device: Mesa DRI Intel(R) Iris 6100 (Broadwell GT3)
OpenGL Driver: 3.0 Mesa 18.1.9
OpenGL Vendor: Intel Open Source Technology Center
Swap Interval: VSync

They do work on another machine (CentOS) though:
> cat /etc/centos-release
CentOS Linux release 7.5.1804 (Core)
> cargo run --example texture
   Compiling tetra v0.1.2 (/mill3d/users/jan/git/github/tetra)
    Finished dev [unoptimized + debuginfo] target(s) in 4.58s
     Running `target/debug/examples/texture`
OpenGL Device: Quadro 4000/PCIe/SSE2
OpenGL Driver: 4.6.0 NVIDIA 390.59
OpenGL Vendor: NVIDIA Corporation
Swap Interval: VSync

Thank you for the report! I'll look into it further this evening - my work laptop has Intel integrated graphics too, so hopefully I'll be able to reproduce the issue on there. Failing that, might need to spin up a Debian VM or something...

I wasn't able to replicate on my laptop, but I think I understand why - from the looks of your log output, it looks like your drivers might be using OpenGL 3.0? I've been targeting 3.3, so perhaps I've used an incompatible feature somewhere... I'll try to force it into 3.0 mode on my machine and see if it breaks :)

Dang, no luck - I set the context version to 3.0 on my laptop, and it still seems to work fine for me. Perhaps it's something to do with the Mesa drivers?

Let  me try something on my side and I will report back ...
# maybe this will help?
apt-get upgrade mesa-common-dev

I upgraded a couple of things (Mesa + Intel Graphics related) but it didn't fix the problem:
OpenGL Device: Mesa DRI Intel(R) Iris 6100 (Broadwell GT3)
OpenGL Driver: 3.0 Mesa 18.2.5
OpenGL Vendor: Intel Open Source Technology Center
Swap Interval: VSync

Anyway, don't worry, I can work on the CentOS box. Close if you want ...

Strange! Thanks for the report, anyway - I'll close it for now, but hopefully I'll get chance to play around on Debian at some point and see if I can replicate.

I'm re-opening this, as @VictorKoenders seems to be having a similar issue on Ubuntu.

Ubuntu 18.10
libsdl2-dev: 2.0.8+dfsg1-4ubuntu1
OpenGL Device: Mesa DRI Intel(R) UHD Graphics 620 (Kabylake GT2)
OpenGL Driver: 3.0 Mesa 18.2.2
OpenGL Vendor: Intel Open Source Technology Center
Swap Interval: VSync

Also doesn't seem to work with sdl2's bundled feature. I fear it's an openGL issue.

@VictorKoenders My suspicion is that the issue might be due to me using #version 330 stuff in my GLSL - could you try running the examples on the simplified-glsl branch that I've just pushed?

That seemed to fix it for me!

A friend found this resource:  https://stackoverflow.com/a/21567167

Ah, the joys of OpenGL...
Anyway, the shader changes don't add any extra complexity/lose us any functionality, so I've merged them into master :)


[Bug]: kwargs not being used in CondensePlusContextChatEngine

Bug Description
The CondensePlusContextChatEngine.from_defaults accepts **kwargs but it is not used anywhere in the method: https://github.com/run-llama/llama_index/blob/254c683afb505f89f390f0b29d9cc120c53c6aeb/llama-index-core/llama_index/core/chat_engine/condense_plus_context.py#L139. I think it should be passed into https://github.com/run-llama/llama_index/blob/254c683afb505f89f390f0b29d9cc120c53c6aeb/llama-index-core/llama_index/core/chat_engine/condense_plus_context.py#L149-L161 at the very least?
Version
0.10.65
Steps to Reproduce
See code references in bug description.
Relevant Logs/Tracbacks
No response

Its only there because index.as_chat_engine abuses kwargs between the retriever and chat engine
Its fine that its not used -- its not needed anywhere afaik

I wanted to use it for passing in a callback_manager without needing to define a global one via Settings.callback_manager while using from_defaults. I thought maybe the **kwargs was intended to allow such a thing.

Nope. Attach the callback manager to your llm and your index/retriever (and possibly the embedding model if using one)


Unable to create Azure VM

Rancher versions:
rancher/server or rancher/rancher: v1.6.18
rancher/agent or rancher/rancher-agent: v1.2.10
Docker version: 1.12.6
Operating system and kernel: Ubuntu 16.04.4 LTS 4.13.0
Type/provider of hosts: Azure
Setup details: HA rancher, external DB
Environment Template: Kubernetes
Steps to Reproduce:
1 - Infrastructure - Add Host with these settings:

Provider: Azure
Environment: AzurePublicCloud
Region: Central US
Image: canonical:UbuntuServer:16.04.0-LTS:latest
VM Size: Standard_DS5_v2 (16 cores, 56gb RAM)

Results
Error in driver during machine creation: compute.VirtualMachinesClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidParameter" Message="Unable to create the VM 'kube-test' in availability set 'elk1-kube-avset' because the requested size Standard_DS5_v2 is not available in the current hardware cluster. The available sizes are: Standard_DS1,Standard_DS2,Standard_DS3,Standard_DS4,Standard_DS11,Standard_DS12,Standard_DS13,Standard_DS14.\r\nRead more on VM resizing strategy at `https://aka.ms/azure-resizevm."
Note:
I'm able to launch Standard_DS5_v2 vms within the same region using the azure portal panel.
The same happens with Standard_D8s_v3 (8 cores, 32gb RAM) or Standard_D16s_v3 (16 cores, 64gb RAM)

The Docker Machine driver creates an availability set to create the virtual machines. This is different than creating an instance through the portal. Azure is saying it doesn't have enough capacity to launch it in an Availability Set (at least not the instance you are requesting). See https://docs.docker.com/machine/drivers/azure/#notes for the Machine driver.
I don't think this is a bug as Azure is replying that it has no capacity, let me know if you think there is something wrong in the integration.


Implement dAg by project

Why:

We need a mechanism that attributes dAg by project and to the platform itself

This change addresses the need by:

Implementing said mechanism
Improving some elements in the UI
cleaning up unused components & code

@DavideSilva cleaned up, thanks 🙇


Create BaseDropOriginalMixin Class

As per ticket here

Added mixin class for transformers that take 'drop_original' argument.
Added this class to mixin script as per weight class mixin script created during test refactor.
Added associated tests to the base tests file rather than creating a new script specifically for this (as per weight class mixin also)
Updated transformers to inherit from new mixin class where required
Updated test files to inherit from new mixin test class where required and where refactor has taken place

Linting errors caused by resolution of merge conflicts, will set up another


banDevice does not ban any devices

Querying available devices I get Available devices: [0,1]
When I try:
CudaEnvironment.getInstance().getConfiguration().banDevice(0)
I get in nvidia-smi output:

And when I try CudaEnvironment.getInstance().getConfiguration().banDevice(1) I get a similar result in nvidia-smi output:

Finally, if I ban device 1 and then print available devices I get Available devices: [0] which means that it is modifying the device list, but not actually taking effect.

Interestingly, if I do the following: .banDevice(0) then I get Available devices: [0].

Is there a way to set cuda visible devices before we start a process? Would that environment variable take precendent during a process?

before we start a process - we should just use CUDA own env variables. But that env variable won't have any effect after app is running.


Internal directory structure

The library keeps its data in ~/.phy by default. The user shouldn't have to deal with this directory in normal circumstances, but they could do it for advanced uses or troubleshooting.
What to store
User-defined settings
Defined in one or several Python files editable by the user. Not edited by the library.

Global
Per data

Examples:

marker size in the feature view (global)
scale factor in the waveform view (per data)
spike detection/clustering default parameters

GUI settings
Defined in an automatically-generated text. Not edited by the user.

Global
Per data

Examples:

placement of the widgets in the Qt GUI (this is a format internal to Qt)
size and possibly positions of the VisPy views
recently-opened files

Clustering data
Per data.

cluster store
clustering history (not implemented yet)

Directory structure
In ~/.phy/:
global/
    clustering_gui  # GUI options
    settings.py
my-dataset-1/  # this is a filename
    cluster_store/
        0001.h5  # one file per cluster
        ...
    clustering_gui  # GUI options
    settings.py

I'd go with something like the following:
##Conventions
EXPBASE = experiment123
EXPFOLDER = /path/to/experiment/

Global data in ~/.phy/
Per-experiment data in EXPFOLDER/EXPNAME_phy (so you can have multiple experiments in one folder - also EXPNAME before _phy so you can do things like `mv EXPNAME* /new/path/ in one command)

Examples:

marker size in the feature view (global)
scale factor in the waveform view (per data)
spike detection/clustering default parameters

GUI settings
Defined in an automatically-generated text. Not edited by the user.

Global
Per data

Examples:

placement of the widgets in the Qt GUI (this is a format internal to Qt)
size and possibly positions of the VisPy views
recently-opened files

Clustering data
Per data.

cluster store
clustering history (not implemented yet)

Directory structure
In ~/.phy/:
gui_settings.py # GUI options
default_params.py # KlustaKwik / spike detection default options etc
phy_state # Qt last view state, recently modified files, size of views etc - AUTOGEN

In EXPFOLDER/EXPBASE_phy:
params.py # KlustaKwik / spike detection options etc
exp_state # Qt view internal savestate (waveform view scaling, channel height etc) - AUTOGEN
[optional, user-created] gui_settings.py # override default GUI settings (reduce spike sample size for huge datasets etc etc)
cluster_store/
      001.phycluster
      002.phycluster


Corrected a few typos in dispatcher comments.

Stumbled across this during a pdb journey.

merged in c68ce18712f5896fa9ab95afd69d590212f3c398, thanks.


Fix service worker

Fixes #24. Looks like it's just the buggy react-scripts rc version. While I was at it I also updated to the newest react 17 version.

Huh, the netlify preview is complaining. It worked on my computer, I'll look at it

it's not working actually. is it okay if I add the React imports back? that's the only way out of this...

Getting this error on the preview. And React 17 is now officially released. Not using the RC versions now

I said it was broken. The react-scripts was rc, not the react.

Okay. Did you put the React Imports back? I think react-scripts needs to be at 4.0.0 to use JSX Transform.

You're right. The SW issue is coming after I updated to React 17.  I think you should put back the React imports because you changed the version of  react-scripts

Alright, will do

Alright, will do
Maybe you want to report this to the react bug thing?

Sure! Thanks!

done

nope wait

@Steffan153 It does not use React. But it's imported 🤔

It complains if I don't add it and it complains if I add it. .

There we go. It worked now

It complains if I don't add it and it complains if I add it. .

Let's leave it as it is for now, I will create an issue on create-react-app about it. Should I close it?

you could merge it, idk
It was basically like "Hey!!!!!! We need react imported for this to work!!!!!!!!!!!!!!!!" but then when you add it eslint comes in like "nobody's using react"

you could merge it, idk
It was basically like "Hey!!!!!! We need react imported for this to work!!!!!!!!!!!!!!!!" but then when you add it eslint comes in like "nobody's using react"

lol. I'll merge it. But still React which is imported is not being used. Thanks!

well it's being used and not being used..... idk

well it's being used and not being used..... idk

It will be hopefull fixed when react-scripts 4.0.0 comes out


Remove distinction between "automatic" execution (macros) and "non-automatic" (bind-key)

When running operations, we have an automatic flag which is set to true if executing from macros, and false when executed from regular bind-key keypresses.
The new bind option suggested in https://github.com/newsboat/newsboat/issues/1165 has some similarities to both macros and the regular key bindings, so it was not directly clear if automatic should be true or false for these new bindings.
I decided to remove the automatic keyword altogether.
Almost all behavior is kept the same, I've tried to remark on behaviour changes in the individual commit messages.
This resolves one of the TODOs of https://github.com/newsboat/newsboat/pull/2364.

coverage: 59.763% (+0.1%) from 59.666% when pulling 04dab2c434473744c8498e67922516280e480219 on dennisschagt:remove-automatic-flag into 7345f152966f57bbf5fd8c5d319b878c96d75c4e on newsboat:master.

I didn't realize that in a macro, an operation will be ignored if it lacks some arguments, and the fact that quit ignores confirm-exit is just bonkers. Yet, someone might rely on this, i.e. they have an incomplete operation in their macro and that doesn't bother them because it does nothing. This PR would be a breaking change for them.
I'd keep this flag until we drop the "macro" concept, which should happen in the next major release once bind is ready to fully replace bind-key and macro. Until then, bind can specify automatic = false just like bind-key does; it's okay because bind is new and there is no pre-existing configs to break :) And its behaviour won't change when we drop the flag.
What do you think? Is there something I'm missing that made you pick this particular approach?
Regardless of what happens to this PR:

I think the docs for bind should highlight this difference in behaviour as compared to macro;
(if we end up closing this PR) c51cabb77fd62c2dfa17ee1b080ac4c6ca960849 is a very good observation and I'd like to have it anyway.

What do you think? Is there something I'm missing that made you pick this particular approach?

I'm fine with keeping automatic around for a while for the few places where it could break the config of users.
However, I think it would be nice to merge commits c51cabb7 and 74d41b28 already because those would make the new bind command a lot more useful.
For example, it looks like configuring bind v articlelist,article set browser mpv ; open-in-browser would not work without the following change (given that bind would set automatic = false):
https://github.com/newsboat/newsboat/pull/2507/commits/74d41b28fb8a8ce4c334384a43548018204db021#diff-58dfeee21458ad853fb41aa977822a6eeb698e655e6b7703ea03905e139386dfL114-R134
Another one I think would be nice is to be able to select a hardcoded or predefined filter without going through the filter dialog:

https://github.com/newsboat/newsboat/pull/2507/commits/c51cabb77fd62c2dfa17ee1b080ac4c6ca960849#diff-1551378fbd86d2c7065b34517b3d6746dc86e657464942650be7078863458be3L484-R495
https://github.com/newsboat/newsboat/pull/2507/commits/74d41b28fb8a8ce4c334384a43548018204db021#diff-d5a1b4371967aac8adde72ce6f97d087e4f8d305f7176749ecdaf22d2ad62844L641-R637

didn't realize that in a macro, an operation will be ignored if it lacks some arguments [..]. Yet, someone might rely on this, i.e. they have an incomplete operation in their macro and that doesn't bother them because it does nothing. This PR would be a breaking change for them.

It might change the behavior, but all of those kinds of changes (in commit 74d41b2) would start asking the user for input (via the qna system or the filepicker dialog). I think that makes it a very safe change (won't cause any potentially destructive actions).

Ah, I see. Bummer :) I agree that the change is safe, but a person who's used to a macro that Just Works will get really confused if the macro started asking questions.
I'm on vacation for the next week, maybe an idea will strike me and I'll know how to have the cake and eat it too :) I'll write back next weekend.

I think we should replace this flag with an enum then. Initially it'd have two values, say, Macro and KeyBind, thus totally equivalent to the flag. When we introduce bind, we'll add a third value and implement the third behaviour. This will create a bit of code duplication for a while, but all of it will be gone when we remove macro and bind-key.
How does that sound?

How does that sound?

That actually sounds like a great idea!
I will see if I can implement it later this week.

Obsoleted by https://github.com/newsboat/newsboat/pull/2555
Thanks again for the great idea @Minoru!


[build-script] Add to default LLDB CMake variables on MacOS

LLDB's CMake build forces these values when building on MacOS. I would like to change LLDB to not force these values for these variables, so setting them in the build invocation seems like the right thing to do.
cc @weliveindetail @dcci @compnerd

Well, in general, making the configuration explicit is a good thing.

@swift-ci please test and merge


Update OpenGLContext_Shared.swift

Update to Swift 3-ready syntax.

I'll hold off on this for the same reason as pull request #4, but I should be able to swap this out soon if I'm reading some of the updates correctly on the Linux ARM packages.


[Objc] Filter devices that are not availabe for debug. Fixes #575

Landing PR since the failures are unrelated to this changes.


Multiline does not wrap to size of the field

If you were to have a large body style input, that in the form itself would wrap multi line, flattening makes it all one line and center to the body of the field.

same problem


[edge] Issue swapping from single to 8-channel pipette

Select 3 wells in a column for single pipette step. Switch to multi channel. Now "3" is in the columns field but when you click and see labware the whole column has been selected. This should have reset well selection.

@howisthisnamenottakenyet I'm having trouble replicating this - for me the well selection clears out when I go single -> multi. Could this be from a specific labware, or when the form is in a certain state? Definitely important to not allow this to happen!

I can't replicate in moveLiquids (Transfer) form, but confimed this is an issue with the Mix form. Should be fixed in #3157


[Popover] the "popover component is too tall" warning shown for 100% height

Setting marginThreshold to 0 and creating a full-screen Popover by also tweaking its Paper styles results in a "Material-UI: the popover component is too tall." warning printed to the console.

[x] This is not a v0.x issue.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior 🤔

Warning shown only when total height of Popover exceeds the available viewport height and not when the height is equal to the available viewport height.
Current Behavior 😯

Warning shown when popover height === client area height, i.e. for true full-screen popovers.
I am pretty sure that the problem is this line of code in Popover.js:
https://github.com/mui-org/material-ui/blob/next/packages/material-ui/src/Popover/Popover.js#L179
The condition should say elemRect.height <= heightThreshold instead of elemRect.height < heightThreshold because heightThreshold is inclusive.
Steps to Reproduce 🕹

Link: https://jsfiddle.net/5oupb4d2/2

Run the Fiddle
Open dev tools and observe warning in console

Context 🔦

Trying to have popovers that behave in a special way for small viewport sizes (i.e. mobile devices) and take up the whole screen as opposed to the 16px margin threshold thus losing a total of 32px of screen estate in both directions.
Your Environment 🌎

Tech
Version

Material-UI
v3.9.2

React
16.8.4

Browser
any

@ignitenet-martynas Your analysis of the situation is correct. I agree with the suggested changes:
--- a/packages/material-ui/src/Popover/Popover.js
+++ b/packages/material-ui/src/Popover/Popover.js
@@ -178,7 +178,7 @@ class Popover extends React.Component {
     }

     warning(
-      elemRect.height < heightThreshold || !elemRect.height || !heightThreshold,
+      elemRect.height <= heightThreshold || !elemRect.height || !heightThreshold,
       [
         'Material-UI: the popover component is too tall.',
         `Some part of it can not be seen on the screen (${elemRect.height - heightThreshold}px).`,

Do you want to submit a pull request? :)

@oliviertassinari  I can pick it up if no one else is on it


Android skips onLoadStart event for PDF links

Bug description:
I want to handle PDF links inside webview so I could intercept such links, stop loading webview and open PDF in Modal. It seems like react-native-webview for Android's implementation "skips" onLoadStart event and fires onLoadProgress only and actually has previous url in syntheticEvent param of onLoadProgress callback. Therefore it's not possible to handle PDF links and pdf download starts immediately via system browser.
To Reproduce:
Clone https://github.com/mir4a/rn-webview-android-pdf-issue repo. It is clear create-react-native-app with 2 additional modules installed: rn-pdf-reader-js and react-native-webview. After you clone the repo run yarn install and npx pod-install and try this app in android yarn android and ios yarn ios.
Expected behavior:
onLoadStart event fires with actual PDF link in SyntheticEvent
Screenshots/Videos:
Android
https://user-images.githubusercontent.com/618900/105712669-68fafe00-5f1a-11eb-8752-82d8b559d8de.mov
iOS
https://user-images.githubusercontent.com/618900/105712689-6f897580-5f1a-11eb-9d07-48014ec553f0.mov
Environment:

OS: macOS
OS version: Catalina 10.15.7
react-native version: 0.63.4
react-native-webview version: 11.2.0

Probably similar to #1074 issue

Probably similar to #1074 issue

I think implementation of onLoadResource would create a nice workaround for such issues. Found closed ticket #1645 with similar suggestion.

I think implementation of onLoadResource would create a nice workaround for such issues. Found closed ticket #1645 with similar suggestion.

Actually found a workaround using onShouldStartLoadWithRequest

Actually found a workaround using onShouldStartLoadWithRequest


Support Spring Boot in Aperture Java  Instrumentation Agent

Tasks suspended - servlet Filters and provided examples should be enough for now