[WIP]training with coodbook loss

TODO:
Ner future:

dataloader and augmentation
quantizer training
generating manifest with codebook indices

Further experiments:

different number of code books
different layer of memory embeddings
different weight of codebook loss
frame-rate mismatch between teach model and student model
use wav2vec as a teach model

@zhu-han Current results show that training with code-book could converge faster and better.
Here is the result training with librispeech-clean-100h dataset.

epoch
baseline wer
+ code book loss

1
73.38
94.24

2
57.24
79.2

3
45.32
66.85

4
36.24
57.71

5
29.66
50.97

6
24.51
43.71

7
21.11
37.85

8
18.89
31.45

9
17.74
27.65

10
16.04
24.65

11
15.55
22.33

12
14.64
20.23

13
14.28
18.52

14
13.42
17.86

15
13.28
16.48

16
12.87
15.92

17
12.83
15.6

18
12.12
14.97

19
12.39
14.85

20
12.1
14.13

21
11.74
13.9

22
11.49
14.04

23
11.3
13.62

24
11.24
13.54

25
10.86
13.44

26
10.84
13.11

27
10.84
12.88

28
10.98
12.74

29
10.69
12.6

30
10.66
12.77

31
10.53
12.65

32
10.27
12.46

33
10.28
12.15

34
10.2
12.19

35
10.02
12.27

36
10.05
12.1

37
9.98
12.2

38
9.78
11.82

39
2.9
11.91

40
9.87
11.91

41
9.72
11.73

42
9.79
11.87

43
9.75
11.15

44
9.57
11.76

45
9.34
11.47

46
9.14
11.21

47
9.16
10.95

48
9.11
11.01

49
8.89
11.3

50
8.8
11.1

51
8.85
11.19

52
8.85
11.2

53
8.65
10.71

54
8.65
10.78

55
8.7
10.86

56
8.72
10.76

57
8.63
10.7

58
8.62
10.49

59
8.67
10.54

60
8.83
10.78

61
8.62
10.17

62
8.71
10.38

63
8.58
10.17

64
8.67
10.35

65
8.8
10.11

66
8.76
10.04

67
8.7
10.07

68
8.69
10.03

69
8.64
10.21

70
8.75
9.57

71
8.59
9.75

72
8.71
9.56

73
8.62
9.84

74
8.86
9.68

75
8.74
10.0

76
8.79
9.54

77
8.84
9.46

Now three directions have been tried:

direction
teacher model
student mode
training data

1
icefall released model trained with 960h
icefall model
clean-100h

2
icefall released model trained with 960h
icefall model
full libri 960h

3
wav2vec model
icefall model
clean-100h

All following results are on test-clean with ctc-decoding.
Conclusions of each direction:
Results of direction 1:
The teacher model help training coverging faster and finally got a much lower wer(around 9.18% vs. 5.9%).
Results of direction 2:
The teacher model help training converging faster. But finally, the results is not significant better than baseline.
Results of direction 3:
Failed to obtain good results, though wav2vec2 model got a much lower wer than icefall released model with only ctc-decoding.

model
wav2vec2
icefall released mode

wer
1.86%
2.93%

To reproduce the 1.86% result, Just run command:
python conformer_ctc/wav2vec_decode.py

Link of wav2vec model used in this exp: https://huggingface.co/facebook/wav2vec2-large-960h-lv60-self
Link of icefall released model used in this exp: https://huggingface.co/csukuangfj/icefall-asr-librispeech-conformer-ctc-jit-bpe-500-2021-11-09
Details results of direction 1:
The teacher model help to converge faster and finally got a much lower wer(around 9.18% vs. 5.9%).

epoch
baseline 1.e
our trained model 2.c.i
our trained model 2.b.v

memory
no
last
last

predictor
no codebook
(old) joint predictor
power ful

weigth
0.0
9.3
0.3

max-duration
100
100
100

gpus
3
3
3

num codebooks
4
4
16

status
(done)
done
done current best

0
90.04
100.0
99.5

1
64.72
64.18
63.6

2
47.81
41.91
44.12

3
36.94
28.10
29.17

4
28.29
21.53
20.83

5
25.28
17.54
16.86

6
21.3
15.14
14.42

7
19.42
14.01
13.38

8
18.35
13.43
12.65

9
17.37
12.94
12.12

10
16.41
12.07
11.17

11
16.07
11.71
10.85

12
15.39
11.2
10.5

13
15.15
11.09
10.1

14
14.68
10.66
9.84

15
14.37
10.54
9.4

16
14.67
10.7
9.58

17
13.82
10.09
9.18

18
13.68
9.7
8.71

19
13.39
9.62
8.62

20
13.14
9.39
8.58

21
12.96
9.11
8.14

22
12.4
8.87
8.15

23
12.08
8.68
7.86

24
11.72
8.45
7.64

25
11.39
8.22
7.47

26
11.45
8.12
7.29

27
11.2
7.88
7.29

28
11.05
7.84
6.93

29
10.82
7.86
6.92

30
10.94
7.63
6.81

31
10.92
7.52
6.93

32
10.97
7.6
6.71

33
10.51
7.59
6.66

34
10.74
7.52
6.78

35
10.32
7.45
6.57

36
10.36
7.5
6.7

37
10.32
7.39
6.56

38
10.1
7.29
6.61

39
10.04
7.25
6.47

40
10.19
7.29
6.44

41
10.01
7.23
6.36

42
10.0
7.24
6.47

43
9.85
7.08
6.38

44
10.01
7.06
6.43

45
9.89
7.06
6.4

46
9.98
6.98
6.29

47
9.75
6.96
6.26

48
9.79
7.06
6.42

49
9.71
6.99
6.24

50
9.8
7.09
6.18

51
9.77
7.01
6.08

52
9.69
7.11
5.96

53
9.59
7.07
6.04

54
9.47
7.01
6.18

55
9.64
7.0
6.08

56
9.68
6.95
6.07

57
9.68
6.94
6.08

58
9.72
6.89
6.03

59
9.38
6.89
5.99

60
9.55
6.9
6.02

61
9.61
6.9
5.97

62
9.43
6.88
6.0

63
9.55
6.88
5.97

64
9.4
6.93
6.06

65
9.54
7.0
5.94

66
9.49
6.81
6.06

67
9.43
6.86
6.07

68
9.33
6.89
5.97

69
9.14
6.79
5.87

70
9.21
6.73
5.91

71
9.07
6.8
5.95

72
9.17
6.88
5.8

73
9.34
6.92
5.93

74
9.18
6.96
5.83

75
9.41
6.87
5.79

76
9.21
6.79
5.97

77
9.18
6.77
5.83

Detail Results of direction 2:
The teacher model help training converges faster. But finally, the results is not significant better than baseline.

epoch
full libri baseline
full libri  2.e.ii

memory
no
last

predictor
no codebook
powerful

weigth
0.0
0.3

max-duration
200
270

gpus
4
3

valid batch duration
800=4*200
810=3*270

batches per epoch
13500
13200

num codebooks
0
16

status
(done) 2.5 hours/ epoch
training 4 hours/ epoch

0
31.96
26.89

1
13.09
9.97

2
9.41
7.67

3
8.29
6.89

4
7.67
6.47

5
7.0
6.08

6
5.75
5.50

7
5.45
5.12

8
5.13
4.76

9
4.81
4.49

10
4.68
4.37

11
4.46
4.35

12
4.38
4.17

13
4.4
4.12


build: Consume latest go-mod-bootstrap with RemoveHosts flag

If your build fails  due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/edgex-go/blob/main/.github/Contributing.md
PR Checklist
Please check if your PR fulfills the following requirements:

[x] I am not introducing a breaking change (if you are, flag in conventional commit message with BREAKING CHANGE: describing the break)
[x] I am not introducing a new dependency (add notes below if you are)
[ ] I have added unit tests for the new feature or bug fix (if not, why?) *N/A
[ ] I have fully tested (add details below) this the new feature or bug fix (if not, why?) N/A - build and unit tests suffice*
[ ] I have opened a PR for the related docs change (if not, why?) N/A

Testing Instructions

New Dependency Instructions (If applicable)

Codecov Report

Merging #4677 (f4f0f76) into main (e4ab1ec) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@            Coverage Diff             @@
##             main    #4677      +/-   ##
==========================================
- Coverage   41.73%   41.73%   -0.01%
==========================================
  Files         105      105
  Lines        9775     9777       +2
==========================================
  Hits         4080     4080
- Misses       5346     5348       +2
  Partials      349      349

Files Changed
Coverage Δ

...rnal/security/bootstrapper/command/flags_common.go
0.00% <0.00%> (ø)


Custom PK name for owned type not recorded in ModelSnapshot

I'm currently using the Npgsql provider with EF Core 2.0 in an ASP.NET Core application.
I've converted all table/column/relationship names to snake case in order to fit the Postgres convention and to make it easier to write native statements in code.
The migrations for this generate fine, but the owned type primary key isn't recorded in the ModelSnapshot, so every time I generate a new migration, it attempts to rename the PK from PK_foo to pk_foo, even though it already is pk_foo in the database itself. This migration will obviously fail unless the offending line is removed.
Is there any way to override this? Or would the workaround be to make it not an owned entity?
The entity classes are:
public class Foo: BaseEntity
{
    // other properties

    public Foo()
    {
        Bar = new Bar();
    }
}

public class Bar
{
    public byte[] Data { get; set; }
}

This is the partial OnModelCreating code:
builder.Entity<Foo>()
   .OwnsOne(t => t.Bar)
   .ToTable("bar")
   .OnDelete(DeleteBehavior.Cascade);

foreach (var entity in builder.Model.GetEntityTypes())
{
   entity.Relational().TableName = entity.Relational().TableName.ToSnakeCase();

   foreach (var property in entity.GetProperties())
   {
   	property.Relational().ColumnName = property.Name.ToSnakeCase();
   }

   foreach (var key in entity.GetKeys())
   {
   	key.Relational().Name = key.Relational().Name.ToSnakeCase();
   }

   foreach (var key in entity.GetForeignKeys())
   {
   	key.Relational().Name = key.Relational().Name.ToSnakeCase();
   }

   foreach (var index in entity.GetIndexes())
   {
   	index.Relational().Name = index.Relational().Name.ToSnakeCase();
   }
}

This is the migration created:
migrationBuilder.CreateTable(
	name: "bar",
	columns: table => new
	{
		foo_id = table.Column<int>(nullable: false),
		data = table.Column<byte[]>(nullable: true)
	},
	constraints: table =>
	{
		table.PrimaryKey("pk_bar", x => x.foo_id);
		table.ForeignKey(
			name: "fk_bar_foo_foo_id",
			column: x => x.foo_id,
			principalTable: "foo",
			principalColumn: "id",
			onDelete: ReferentialAction.Cascade);
});

This is the relevant part of the model snapshot
modelBuilder.Entity("ExampleProject.Foo", b =>
{
	b.OwnsOne("ExampleProject.Bar", "Bar", b1 =>
		{
			b1.Property<int>("FooId")
				.HasColumnName("foo_id");

			b1.Property<byte[]>("Data")
				.HasColumnName("data");

			b1.ToTable("bar");

			b1.HasOne("ExampleProject.Foo")
				.WithOne("Bar")
				.HasForeignKey("ExampleProject.Bar", "FooId")
				.HasConstraintName("fk_bar_foo_foo_id")
				.OnDelete(DeleteBehavior.Cascade);
		});
});

EF Core version: 2.0
Database Provider: Npgsql
Operating system: Windows 10
IDE: Visual Studio 2017 15.3.4

Interesting. There's no way to address the primary key name from the ModelBuilder API.
The only way I can see to configure it is to drop down to metadata:
b1.Metadata.DeclaringEntityType
    .GetOrSetPrimaryKey(b1.Property("FooId").Metadata)
    .Relational().Name = "pk_bar";

Triage: we are holding off on this for now since wider configuration of owned types may either make the existing HasKey available again, or make it more reasonable to add HasKey here. See #1985


About the choice of optimizer

My current work requires me to generate images of dolls as an alternative to photography. The doll itself is nude, and the pictures generated need to reflect body details, so the pictures I provided for training would be more complicated. I found that adamw8bit could not better train FLUX's lora model (it could not be fitted with low learning rate, and the pictures with high learning rate were directly blurred). The prodigy optimizer used in training SDXL before can better get the desired results, can you add prodigy to the trainer?

Prodigy is already present

Prodigy is already present

Any hints on how to use it? Just replacing adamw8bit for Prodigy in the optimizer? No extra special params needed?
Thank you!

   `lr: 1
    noise_offset: 0.1
    lr_scheduler: "cosine"
    optimizer: "Prodigy"
    optimizer_params:
      decouple: true
      use_bias_correction: False
      betas: [0.9, 0.99]
      weight_decay: 0.05`

  This is what Im using, around 2000 steps for 10 images

   `lr: 1
    noise_offset: 0.1
    lr_scheduler: "cosine"
    optimizer: "Prodigy"
    optimizer_params:
      decouple: true
      use_bias_correction: False
      betas: [0.9, 0.99]
      weight_decay: 0.05`

  This is what Im using, around 2000 steps for 10 images

Awesome, thanks a lot!

This OOM for me on 4090.  Rank/Alpha are you using with Prodigy?

Coming over from SDXL I had read some time ago that Prodigy and Adafactor should have an alpha of 1 because of the adaptive learning rate. Network dim I've always run Prodigy on 16-32.
Flux is pretty heavy on the VRAM though and so is Prodigy, and there seems to be some differences in VRAM management from one training program to another. Over on SDXL with my 4090, I have been using Easy Lora Training Scripts with its kohya backend with Prodigy on batches of 12 but using straight kohya I get OOM's above a batch size of 4 and I have yet to be able to train a full checkpoint.
I have a feeling a good portion of Flux is going to remain out of reach of us mere 4090 peasants.


Document how to install new Project SDK

Environment

Documentation version: 7.0

Description of the bug or enhancement
Please add instructions for the new user.
Let's say he just installed IDEA and CUBA plugin.
When he creates a new project, no SDKs are installed for the IntelliJ IDEA.

We can just give link to existing IntelliJ IDEA documentation page

I've checked on Windows and Linux, with IDEA 2019.2. IDEA automatically discovers JDK that are installed into common places (program files, /usr/lib/jvm).
So I think additional explanation is not necessary.


python310Packages.time-machine: 2.8.2 -> 2.9.0

Diff: https://github.com/adamchainz/time-machine/compare/2.8.2...2.9.0
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 224541 run on x86_64-linux 1

2 packages failed to build:

rich-cli
rich-cli.dist

25 packages built:

dooit
dooit.dist
home-assistant-component-tests.bmw_connected_drive
home-assistant-component-tests.netatmo
home-assistant-component-tests.switcher_kis
python310Packages.aioswitcher
python310Packages.aioswitcher.dist
python310Packages.bimmer-connected
python310Packages.bimmer-connected.dist
python310Packages.pyatmo
python310Packages.pyatmo.dist
python310Packages.textual
python310Packages.textual.dist
python310Packages.time-machine
python310Packages.time-machine.dist
python311Packages.aioswitcher
python311Packages.aioswitcher.dist
python311Packages.bimmer-connected
python311Packages.bimmer-connected.dist
python311Packages.pyatmo
python311Packages.pyatmo.dist
python311Packages.textual
python311Packages.textual.dist
python311Packages.time-machine
python311Packages.time-machine.dist


[MLLIB] SPARK-2311: Added additional GLMs (Poisson and Gamma) into MLlib

SPARK-2311 - Added additional GLMs (Poisson and Gamma) into MLlib
implemented PoissonRegressionSGD and GammaRegressionSGD.

I imagine this is too far out of date, and perhaps obsolete given the new ML API coming. Mind closing this PR?

@srowen This work is originally for version 1.0.x and is pretty out-dated.
@xwei-datageek  Xiaokai, I think it's ok to close this PR.
As for modeling count data using regression models, I think SparkR with glm package would be a good solution though I have not get deep into it.


Feature/55 allow importing pre annotated data

Permit sending a typesystem along when uploading a document

Codecov Report

Merging #56 (1425b9b) into dev (8be0795) will increase coverage by 0.04%.
The diff coverage is 94.73%.

@@            Coverage Diff             @@
##              dev      #56      +/-   ##
==========================================
+ Coverage   90.46%   90.51%   +0.04%
==========================================
  Files          15       15
  Lines        1364     1381      +17
==========================================
+ Hits         1234     1250      +16
- Misses        130      131       +1

Impacted Files
Coverage Δ

averbis/core/_rest_client.py
84.98% <90.90%> (+0.09%)
:arrow_up:

tests/test_document_collection.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8be0795...1425b9b. Read the comment docs.


Emacs menu area 'not covered' by theme

Big nord fan here.  Just upgraded a machine from Ubuntu 19.04 to 19.10, and now the menu bar appears to be uncovered by the theme:

Seemingly the 19.04 release has Emacs 26.1 whereas this now happens with 26.3.  Anybody know how I could overcome this?

Hi @eddelbuettel, nice to see you like Nord :smile:
I guess you're running Emacs in GUI mode. I have to admit that I've never really paid attention to the menu bar color. My first guess would be that it is part of the window so the running window manager (latest Ubuntu version using GNOME so it's Mutter) and the style of based on the used GTK3 theme, but I could also be completely wrong.
I'll create VirtualBox to test the menu with Ubuntu 19.04.

I do :)  In gnome-terminal / tmux and Emacs.  I later realized that this may have something to do with Ubuntu too.  Case in point is that their default them also bit the Atom editor where the menu bar is/was unavailable / invisible for at least a year (and two releases) if not more -- can probably find links.
And while I am a loooooong time x11 user I have no idea how to find the resource name of that section.  A quick poke at editres revealed nothing.   So whatever you can come up with I'd gratefully use (and test and re-test ...).

A quick scan of the changelog for version 26.1 to 26.3 includes a change that might be related to your problem:

Building Emacs with the '--with-xwidgets' option now requires WebKit2.
To build Emacs with xwidgets support, you will need to install the
webkit2gtk-4.0 package; version 2.12 or later is required.
(This change was actually made in Emacs 26.1, but was not called out
in its NEWS.)

I don't know for which elements WebKitGTK is used by Emacs, but maybe it's used to style some GTK3 components. The package is available from the official Ubuntu repositories.

Could well be--I just use the default binary.  On a lurk, I install emacs-lucid, and it does of course not have the problem.  (But makes me feel it is 1998 all over again, but that is a different issue ;-) )

Actually, and one thing I just noticed -- it is also the same under the older (26.1) Emacs when running in console mode for example in tmux.  Screenshot from Emacs/Magit on an (older) checkout of Nord:

So how do we get to that menu line?

I think it may not be you but "them", i.e. an Ubuntu theme issue:  https://www.linuxuprising.com/2019/10/how-to-get-dark-gnome-shell-menus-and.html

I think I will close this. It very appears "by design" in Ubuntu and e.g. goes away when selecting a dark theme via the Tweak tool.  And by now ... I have gotten used to it :)    All still good with Nord just about everywhere:  gnome-shell, tmux/byobu, emacs, ...


Show a progress bar when fetching pack indices for choice set rule elements

Currently loading all indices for e.g. the Versatile Heritage can take more than 20 seconds on slower connections with no indicator why the prompt isn't opening.
Also removes JQuery from the Progress class.

Maybe we should use the core progress bar for this? I know Crucible is using it when loading up the talent tree.

Crucible is utilizing the TextureLoader class  that uses SceneNavigation.displayProgressBar which does most of what our Progress class was doing. I've removed all unnecessary code from Progess.


Update items.toml

New activity added

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[Question]Issue with harvester installer

Hi, i'am trying to install harevster 0.2.0 on ProLiant XL170r Gen9, but after confirm of installation i have an issue. At the moment i cannot find any solution about that, any advise about ? Everything on server is up to date. Also there is internet connection on that. Of course i try to install v0.3.0-preview1, v0.2.0-rc5 with the same results. Also i try to install harvester on other servers DL170e G6, DL165 G7 with the same issue. Should i check something spesific for example on BIOS? I try to boot harvester with two ways, with media/cd through iLO and with USB stick, and also in dd with rufus.

can u please try the https://github.com/harvester/harvester/releases/tag/v0.3.0, thanks


Custom sms provider not working after server restart in super tenant

Describe the issue:
Custom sms provider not working after server restart in super tenant.
How to reproduce:

Add a Custom SMS provider. (check that it works ex. Add a webhook as a customer provider and test that it works by using mobile attribute change verification)
Re start the server and test again and it will not work.

Root cause:
The custom SMS providers are loaded to the system when tenants are loaded through [1]. This loader is not getting called in the server startup so the super tenant custom sms provider is not getting loaded.
Note: SMS otp flow works as it doesn't depend on event streams based publishing and not reliant on the carbon analytics based eventing module
[1] - https://github.com/wso2-extensions/identity-governance/blob/master/components/org.wso2.carbon.identity.tenant.resource.manager/src/main/java/org/wso2/carbon/identity/tenant/resource/manager/TenantAwareAxis2ConfigurationContextObserver.java#L62

This was solved by loading configurations using a startup observer and is not optimal as it is called after the server startup completion. These configurations should be loaded before the server startup completion event. Before we can add the publishers the event streams need to be loaded to the server, current event streams are file based, and it's loading is triggered using a server startup observer which runs on completingServerStartup method (org.wso2.carbon.core.internal.DeploymentServerStartupObserver). There is currently no way to make sure the calling order, therefore this is the only way to guarantee the publishers are loaded after the event streams are loaded.
The file based publisher loading happens from org.wso2.carbon.event.publisher.core.EventPublisherDeployer

This was solved in wso2-extensions/identity-governance#805 by loading configurations using a startup observer and is not optimal as it is called after the server startup completion. These configurations should be loaded before the server startup completion event. Before we can add the publishers the event streams need to be loaded to the server, current event streams are file based, and it's loading is triggered using a server startup observer which runs on completingServerStartup method (org.wso2.carbon.core.internal.DeploymentServerStartupObserver). There is currently no way to make sure the calling order, therefore this is the only way to guarantee the publishers are loaded after the event streams are loaded.
The file based publisher loading happens from org.wso2.carbon.event.publisher.core.EventPublisherDeployer

Created the issue https://github.com/wso2/product-is/issues/19726 to move the sms sending to the new way used in sms otp


Setup mkdocs

This PR sets up an mkdocs site for the emails guides. It also removed the generated svg files from the source and instead generates them on build of the medics site. This requires them to be moved onto a single directory.

With a naive mkdocs serve I don't seem to have the Manufacturing SVGs generated but it's plausible I'm not reproducing the correct build environment. (My mkdocs-build-plantuml-plugin is at v1.7.4.) Just something for another reviewer to look out for.

It may be appropriate to remove the $(1)-diagrams target in the make file.

With a naive mkdocs serve I don't seem to have the Manufacturing SVGs generated but it's plausible I'm not reproducing the correct build environment. (My mkdocs-build-plantuml-plugin is at v1.7.4.) Just something for another reviewer to look out for.

I've moved the diagram changes out to a separate PR, this now only adds mkdocs setup

It may be appropriate to remove the $(1)-diagrams target in the make file.

Thanks, I'll address this in a separate PR with the diagram changes


Fix the-main-function BBE to match the outfile

Purpose
Outfile does not match the output of the code because of a condition.
This PR fixes it.
Also fixes a slight change in output file in functional iteration example.

Codecov Report

Merging #18849 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #18849   +/-   ##
=======================================
  Coverage   19.92%   19.92%
=======================================
  Files          52       52
  Lines        1355     1355
  Branches      208      208
=======================================
  Hits          270      270
  Misses       1072     1072
  Partials       13       13

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2393f41...92c8635. Read the comment docs.


npm install issue

hey,
tried to setup the project for the goodness squad on the 15th
win 10 PC,
found nodejs version in travis config and installed it, npm install not exiting
tried cmd with admin rights, restarting the process, restarting  cmd
any ideas on how to setup the project on PC?

restarting windows resolved the issue -_-
docker container?


Issue & PR templates added

Solving #49
I have added two files in .github directory that are ISSUE_TEMPLATE.md & PULL_REQUEST_TEMPLATE.md

@naveen8801 thank you for your contribution 🥇


Feature Request: Display uncaught errors in the DOM

By accident, I tried to run a 64-bit image in SqueakJS today and stared a few minutes at the loading bar before I opened the Chrome Console and read the "Uncaught Error: 64 bit images not supported yet" message there. It would be nice if these errors could be displayed in the DOM, or using an alert instead, so you cannot overlook them. :-)

Agreed. I'll put it on my to-do list :)


Refactor building of hydro profiles from EIA BA-level shapes to include EIA 923 information

:rocket:

[ ] Is your feature request essential for your project?

Describe the workflow you want to enable
Currently, when building hydro profiles for the HIFLD grid model using prereise.gather.griddata.hifld.data_process.profiles.build_hydro, we use the balancing-authority-aggregated generation shape (after filtering and imputing) to populate the time-series profile for each hydro plant. There is additional information we could incorporate into these profiles however: information from EIA Form 923 which provides monthly net generation on a plant-level resolution.
Context
Looking at the existing time-series outputs of build_hydro and comparing to values from EIA 923, we see pretty significant differences, mostly for the smaller generators+month combinations but some significant differences exist for the larger ones as well.
For 2020 (the most recent year of full final data), the r-squared is 0.895, which seems pretty good! Raw MAPE is about 1200% for the 1,389 matching plants (seems pretty bad), although when we weight this by the generation for each (plant + month) combination, the weighted MAPE is about 40%: much better, but still not great. Part of this could be that there is a large chunk of data missing for CISO in this year, so CISO plants use an averaged profiles of all the other BAs. The following plots all show the same information, with different axis scales:

For 2021 (final data not yet available), there are only 135 matching plants (only those reporting monthly). The raw MAPE is worse than 2020, nearly 1600%, but the weighted MAPE is better, about 34%, and the r-squared is 0.897.

Code to reproduce (with the hifld branch of PreREISE checked out):
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy.stats import linregress
from prereise.gather.griddata.hifld.data_process.profiles import build_hydro
from prereise.gather.griddata.hifld.orchestration import create_grid

# Define a function to report results for an arbitrary year
def compare_monthly_generation(hydro_plants, eia_923_df, year):
    def plot_by_month(parsed_eia_923_df, monthly_hydro):
        fig = plt.figure(figsize=(16,12))
        ax = fig.add_subplot()
        for month_num, month_name in months.items():
            plt.plot(
                parsed_eia_923_df.loc[month_num, common_plant_codes],
                monthly_hydro.loc[month_num, common_plant_codes],
                '.',
                label=month_name,
            )
        plt.plot([1e1, 2.8e6], [1e1, 2.8e6], 'k--')
        plt.legend()
        plt.xlabel("EIA monthly generation (MWh)")
        plt.ylabel("build_hydro monthly generation (MWh)")
        return ax

    # Build time-series hydro profiles from Balancing authority shapes
    hourly_hydro = build_hydro(
        hydro_plants.set_index("Plant Code"),
        eia_api_key=eia_api_key,
        year=year,
    )
    # Parse hourly to monthly
    monthly_hydro = (
        hourly_hydro.groupby(hourly_hydro.index.month).sum()
        * hydro_plants.set_index("Plant Code")["Pmax"]
    )
    # Interpret raw EIA 923 data to (month x plant_code) data frame
    parsed_eia_923_df = (
        eia_923_df
        .loc[
            (eia_923_df["Reported\nPrime Mover"] == 'HY')
            & (eia_923_df["Plant Id"] != 99999),
            ["Plant Id"] + list(eia_923_cols_of_interest)
        ]
        .replace(".", float("nan"))
        .groupby("Plant Id").sum()
        .astype(float)
        .rename(eia_923_cols_of_interest, axis=1)
        .T
    )
    # Identify which Plant Codes match across data sets
    common_plant_codes = set(parsed_eia_923_df.columns) & set(monthly_hydro.columns)
    print(len(common_plant_codes), "matching plant codes")
    # Make plots
    ax = plot_by_month(parsed_eia_923_df, monthly_hydro)
    plt.show()
    ax = plot_by_month(parsed_eia_923_df, monthly_hydro)
    ax.set_xlim(0, 5e5)
    ax.set_ylim(0, 5e5)
    plt.show()
    ax = plot_by_month(parsed_eia_923_df, monthly_hydro)
    ax.set_xscale("log")
    ax.set_yscale("log")
    plt.show()
    # Calculate statistics
    raveled_923 = np.ravel(parsed_eia_923_df[common_plant_codes])
    raveled_hydro = np.ravel(monthly_hydro[common_plant_codes])
    total_length = raveled_923.shape[0]
    ape = np.zeros(total_length)
    for i in range(total_length):
        if raveled_923[i] in {float("nan"), ".", 0}:
            ape[i] = float("nan")
        else:
            ape[i] = abs((raveled_hydro[i] - raveled_923[i]) / raveled_923[i])
    good_data_mask = ~np.isnan(ape)
    mape = np.mean(ape[good_data_mask])
    print(f"{mape=}")
    weighted_mape = (
        np.sum((ape * np.ravel(parsed_eia_923_df[common_plant_codes]))[good_data_mask])
        / np.sum(np.ravel(parsed_eia_923_df[common_plant_codes])[good_data_mask])
    )
    print(f"{weighted_mape=}")
    slope, intercept, rvalue, pvalue, stderr = linregress(
        raveled_923[good_data_mask], raveled_hydro[good_data_mask]
    )
    print("r-squared", rvalue**2)

# Constants
eia_api_key = YOUR_EIA_API_KEY_HERE
months = {
    1: "January",
    2: "February",
    3: "March",
    4: "April",
    5: "May",
    6: "June",
    7: "July",
    8: "August",
    9: "September",
    10: "October",
    11: "November",
    12: "December",
}
eia_923_cols_of_interest = {f"Netgen\n{v}": k for k, v in months.items()}

# Build the HIFLD grid without dropping columns that are irrelevant to PowerSimData
# This step can take a while, depending on what you have cached
# Go grab a cup of coffee while you wait
full_tables = create_grid()
# Read 2020 data, generate 2020 hydro, compare
eia_923_2020 = pd.read_excel(
    PATH_TO_YOUR_LOCAL_EIA_FORM_923_2020_FILE,
    header=5
)
compare_monthly_generation(full_tables["plant"].query("type == 'hydro'"), eia_923_2020, 2020)
# Read 2021 data, generate 2021 hydro, compare
eia_923_2021 = pd.read_excel(
    PATH_TO_YOUR_LOCAL_EIA_FORM_923_2021_FILE,
    header=5
)
compare_monthly_generation(full_tables["plant"].query("type == 'hydro'"), eia_923_2021, 2021)

Describe your proposed implementation, if applicable
build_hydro is refactored to be able to (potentially toggled by a feature flag) scale the hourly BA-level shapes to better represent the monthly energy totals from EIA 923. I believe this information can be retrieved from the EIA's API using the same API key as we use to grab the BA-level hydro generation, or alternatively could be retrieved from an alternate source (see #246). Using a naive scaling factor based on (EIA 923 monthly energy / build_hydro monthly energy) could be problematic since it could result in values which are greater than 1 during some hours. If we use this approach, we'll need to be sure to clip these values to be no larger than 1.
EIA Form 923 also seems to sometimes contain distinct entries for conventional hydro ("Reported Prime Mover" == "HY") vs. pumped storage hydro ("Reported Prime Mover" == "PS") within the same Plant Code entry, and reports the 'fuel consumed' for pumping. We may be able to use this information to identify which plants are pumped storage (and whose profiles can therefore go negative) vs. which plants are conventional hydro only (and should have their profiles clipped to a minimum of 0).

Proposed implementation plan, part 1: refactor build_hydro and add a feature flag (toggle). Goal of the feature is to better represent the monthly energy totals from EIA 923. "Better represent" will be assessed on the basis of R^2 and MAPE, comparing against prior metrics generated by Daniel O. and reported in the issue. The code provided in the issue can be used to directly assess these metrics. Actual values for 2020 will be retrieved with the same API key as we use to grab the BA-level hydro generation. Actual values will also be used to create scaling factors which will be applied to our profiles.
Plan part 2: Identify pumped hydro vs. conventional hydro and clip profile of conventional hydro with minimum value of 0. Assess against R^2 and MAPE to see if this is an improvement. If so, implement this additional improvement.
EIA Form 923 also seems to sometimes contain distinct entries for conventional hydro ("Reported Prime Mover" == "HY") vs. pumped storage hydro ("Reported Prime Mover" == "PS") within the same Plant Code entry, and reports the 'fuel consumed' for pumping. We may be able to use this information to identify which plants are pumped storage (and whose profiles can therefore go negative) vs. which plants are conventional hydro only (and should have their profiles clipped to a minimum of 0).


GoogleARCoreServices plugin could not be found

I am trying to open the CloudARPin project, but the GoogleARCoreServices plugin is missing. I have tried to follow the links to the repo, but find myself at a 404. Where might I find an available download?

@bopangzz thanks, I will try to use the one from the source.
I was wondering if the 404 was due to not having access. I went through those steps a couple times now and github seems to not want to connect on unReals end.

You should get an email to ask you to join Epic's GitHub organization once you connect your github account to your epic account. You should have access once you join the organization.

Thanks, I will keep an eye out for that.

Issue resolved.
The invitation was available on github, email came after I accepted. I was able to download 4.20arcore, build, and load editor via VS with plugin included.
thanks

Glad it works for you! Closing the bug now.


Functional api/identify effect

First of a series of PR to add a functional API according to: https://github.com/py-why/dowhy/wiki/API-proposal-for-v1

Refactor identify_effect to have a functional API

Created BackdoorIdentifier class and extracted the logic from CausalIdentifier to be just a Protocol
Refactor the identify_effect method of BackdoorIdentifier and IDIdentifier to take the graph as parameter
Moved constants into enums for easier type checking
Backwards compatible with previous CausalModel API
Added notebook as demo that CausalModel API and new API behaves the same way

I can see now that the identifiers are actually used in CausalModel (which I missed previously). So it seems we need those classes for backwards-compatibility.
If this is true, I'm wondering: should AutoIdentifier simply delegate to a new auto_identify_effect function and IDIdentifier to a new id_identify_effect function, and then both classes are marked as deprecated. Otherwise, I see the risk that the direction is not clear and new code builds again on these classes instead of the functions.
@amit-sharma That's probably not just a comment for @andresmor-ms, but also a more general question. What do you think?

@andresmor-ms My apologies for being a bit late in the game here. I missed this one.
I have a very general question: What is the idea behind having classes for identifiers opposed to, say, a simple function? E.g. auto_identify_effect(...)
Typically, we want to use objects, when we plan to give them a longer lifecycle. E.g. we want to pass them around where the consumer is not required to know what kind of identifier it is (IDIdentfier or AutoIdentifier). But if our typical usage is
identifier = AutoIdentifier(...)
identifier.identify_effect()

this can be simplified by providing a single call. As an added benefit, a lot less bookkeeping of variables is necessary (in the implementation of the class).
So yea, I'm curious about the usage patterns you have in mind here.

No worries.
The intention of not refactoring this as functions (and keeping the classes) is mainly to keep backwards compatibility with the object-oriented API, I agree that having functions like auto_identify_effect(...) would simplify the code and the API, however there are some parts of the code that reference a CausalIdentifier object (for example the CausalModel keeps track of the identifier that was used), I could dig into what the usage is and check if it can be easily removed as part of this refactor.
As for usage patterns I believe that what we want to achieve is having a single point where you can configure everything (the identify_effect(...) function), in the future this function would have more logic and defaults to guide the user what to use if they don't know what paramters should be set, the general usage would be:
import dowhy

dowhy.identify_effect(
    graph=graph,
    treatment=treatment_name,
    outcome=outcome_name,
    method=AutoIdentifier(...),
)

Just as a note, per my understanding what Peter suggest is to use it like this:
import dowhy

dowhy.auto_identify_effect(
    graph=graph,
    treatment=treatment_name,
    outcome=outcome_name,
    estimand_type=...,
    adjustment_method=...,
)

(We could still have a general identify_effect(...) function with defaults for users that need guidance or don't know which one to use)
@amit-sharma, @emrekiciman, I'd also like to know your opinion on this.

Backwards compatibility is important for a while -- we haven't decided how long but I would think at least 6-12 months.
Yes, general usage would be per below.  For the unfamiliar user, the identify_effect function could call AutoIdentifier as a default.  I.e., like this:
import dowhy

dowhy.identify_effect(
    graph=graph,
    treatment=treatment_name,
    outcome=outcome_name
)

Conceptually, I see the identify_effect() function signature developing into a function that accepts arguments that are related to the causal question being asked: treatment, outcome, whether we care about direct/indirect effects, CATE, etc.  In contrast, the specific identification methods (AutoIdentifier, IDIdentifier, ...) may also accept configuration arguments (how hard to search, whether to return a maximal or minimal conditioning set) that are more about the identification method rather than the causal question being asked.  I expect we'll have more PRs to address this, though.  Right now, would be good to keep this PR focused on the functional refactoring?

@petergtz said:

If this is true, I'm wondering: should AutoIdentifier simply delegate to a new auto_identify_effect function and IDIdentifier to a new id_identify_effect function, and then both classes are marked as deprecated. Otherwise, I see the risk that the direction is not clear and new code builds again on these classes instead of the functions.

Peter, I think deprecating the classes is where want to go.  I'd probably hold off on marking them as deprecated at least until we complete the functional refactoring of the whole API.  I might also suggest not marking them as deprecated until the new API is no longer marked "experimental".  I.e., I don't think we want a user to have to choose between using "deprecated" vs "experimental" APIs :D
Emre

I.e., I don't think we want a user to have to choose between using "deprecated" vs "experimental" APIs :D

@emrekiciman Yes, that's certainly true :-). Also agree with the rest of what you said. It seems like what I'd like to see is that we don't want new library features built heavily on things we plan to deprecate. And that's sometimes hard to keep in mind when plans are made, but their execution takes several months. So comments here would help.
I believe we agreed, we would implement the "legacy" API using the new functional API. That's why I suggested to introduce auto_identify_effect and id_identify_effect and then have the corresponding classes just call these.
Then we could add a comment on the classes saying that "we plan to deprecate them" and "new code should directly depend on corresponding_function_xyz".
I'm fine with the global functionidentify_effect when it provides convenient defaults for e.g. the effect identifier. That will already provide value.

@petergtz said:

If this is true, I'm wondering: should AutoIdentifier simply delegate to a new auto_identify_effect function and IDIdentifier to a new id_identify_effect function, and then both classes are marked as deprecated. Otherwise, I see the risk that the direction is not clear and new code builds again on these classes instead of the functions.

Peter, I think deprecating the classes is where want to go. I'd probably hold off on marking them as deprecated at least until we complete the functional refactoring of the whole API. I might also suggest not marking them as deprecated until the new API is no longer marked "experimental". I.e., I don't think we want a user to have to choose between using "deprecated" vs "experimental" APIs :D
Emre

This is what I'm thinking to move forward:

Refactor the classes to just call a function auto_identify_effect or id_identify_effect (as peter suggested)
Keep the identify_effect() function as entry point with defaults for beginners.
Do not deprecate yet the old API as the new one is not complete, add docs to class to warn that it will be deprecated in the future

What do you think? @emrekiciman @petergtz @amit-sharma

What do you think?

Sounds great! Thanks, really appreciate your effort here, @andresmor-ms.

If this is true, I'm wondering: should AutoIdentifier simply delegate to a new auto_identify_effect function and IDIdentifier to a new id_identify_effect function, and then both classes are marked as deprecated. Otherwise, I see the risk that the direction is not clear and new code builds again on these classes instead of the functions.

I'd like to offer a different perspective. I agree on removing CausalModel class and moving to functional top-level user API because the CausalModel class was just a book-keeping class. For the AutoIdentifier/IDIdentifier classes, here's some reasons to keep them as classes (the same arguments may also apply to estimation classes CausalEstimator and CausalRefuter classes):

Having identifier methods as classes can make it easier for newcomers to add a new identification method. They would simply need to inherit from CausalIdentifier and then implement one method, identify_effect. All the other common methods will be available for free. So far, this class structure has worked really well for external contributors, who have found it easy to come up with PRs for new identification or refutation methods. A nice side-effect is that all the new experimental code is safely wrapped in a class, with low chances of affecting other functionality.
Having a common type for all identifier methods is useful for type-checking the top-level API, dowhy.identify_effect(method=...). Here users can provide any class that subclasses CausalIdentifier. This also helps in setting up basic minimum hygiene (e.g., all identifiers are expected to take in minimum standard parameters).
For the identifier method specifically, it helps us separate the initialization parameters of an identifier from actually calling it. This is helpful when a user wants to use the same identifier repeatedly for multiple effects in a graph (for different treatments and outcomes).
Thinking ahead for estimation where we also have classes for each estimator, it helps us setup a minimum protocol for adding a new estimator (e.g., implementing fit, predict and do methods in a class). Since sklearn methods are class-based, it allows people to easily extend existing sklearn models for causal inference, by simply adding a do method.

More generally, I am thinking about an ideal state where we provide a stable, user-facing API for causal tasks. At the same time, we make it super simple for people to add new underlying methods that support the stable API. In that direction, I feel having underlying methods as classes could be a useful abstraction.

Having identifier methods as classes can make it easier for newcomers to add a new identification method. They would simply need to inherit from CausalIdentifier and then implement one method, identify_effect. All the other common methods will be available for free. So far, this class structure has worked really well for external contributors, who have found it easy to come up with PRs for new identification or refutation methods. A nice side-effect is that all the new experimental code is safely wrapped in a class, with low chances of affecting other functionality.

@amit-sharma I'd like to respectfully disagree with the motivation to have a class for simple code re-use. Using inheritance for code re-use has fallen out of favor for multiple reasons in software design. Why that is, has been explained in multiple other places. The Rust book also describes this in the chapter about OOP. Python allows it, but I wouldn't use the feature unnecessarily. And I think what applies for us, is, that requiring to inherit from CausalIdentifier, to be able to re-use those methods, is actually more burden than just being able to call the plain functions (in the way that @andresmor-ms has implemented it in the latest revision of this PR).

Having a common type for all identifier methods is useful for type-checking the top-level API, dowhy.identify_effect(method=...). Here users can provide any class that subclasses CausalIdentifier. This also helps in setting up basic minimum hygiene (e.g., all identifiers are expected to take in minimum standard parameters).

I think we discussed to introduce type hints for type-checking. That way we wouldn't require a wrapper for this task. Note that I'm not necessarily opposed to the global identify_effect function when it provides a simplified API with helpful defaults over the specialized functions.

For the identifier method specifically, it helps us separate the initialization parameters of an identifier from actually calling it. This is helpful when a user wants to use the same identifier repeatedly for multiple effects in a graph (for different treatments and outcomes).

That's indeed a scenario I could see for certain use cases. That's why I asked if we expect this usage pattern in this comment. Note though: we'd have to change the API of those classes nonetheless: right now, treatment_name and outcome_name are part of initialization, not of identify_effect.

Thinking ahead for estimation where we also have classes for each estimator, it helps us setup a minimum protocol for adding a new estimator (e.g., implementing fit, predict and do methods in a class). Since sklearn methods are class-based, it allows people to easily extend existing sklearn models for causal inference, by simply adding a do method.

Agree with estimators being classes. The usage is slightly different from identifiers though, and we take advantage of polymorphism (opposed to code re-use): by having an interface/Protocol for estimators we can opaquely pass an estimator we got from fitting the estimand to the estimate_effect function without knowing the actual implementation of the estimator.

I don't feel strongly about classes vs global functions, so won't say much about that.
Looking at the current code, I do have a question about how we can be clear in the code about the official and supported API, vs. internal functions.  For example, let's say we want identify_effect, auto_identify_effect and id_identify_effect as top-level functions that users can call directly.  But I'm looking at the code and also seeing top-level functions for identify_nde_effect, find_valid_adjustment_sets, build_backdoor_estimands_dict, identify_mediation_second_stage_confounders, construct_frontdoor_estimand, etc.
So, this is maybe more of a python question.  What's the way we keep (at least most of) these other functions private, hidden, or otherwise clearly outside of the supported API?

So, this is maybe more of a python question. What's the way we keep (at least most of) these other functions private, hidden, or otherwise clearly outside of the supported API?

@emrekiciman https://peps.python.org/pep-0008/#public-and-internal-interfaces provides some guidelines here:

Documented interfaces are considered public, unless the documentation explicitly declares them to be provisional or internal interfaces exempt from the usual backwards compatibility guarantees. All undocumented interfaces should be assumed to be internal.

Also:

To better support introspection, modules should explicitly declare the names in their public API using the __all__ attribute. Setting __all__ to an empty list indicates that the module has no public API.

And finally:

internal interfaces (packages, modules, classes, functions, attributes or other names) should still be prefixed with a single leading underscore.

The latter recommendation is often a bit unpractical in my experience: when you prefix a module or function with _, style checkers or linters often flag this when you import it form another module. The control mechanisms are simply not precise enough in Python. Which is why I'd go with the first 2 recommendations.

Sorry to ask a question on an old MR but I'm trying to understand some of the design direction. I'm having a bit of trouble  understanding:

I can see now that the identifiers are actually used in CausalModel (which I missed previously). So it seems we need those classes for backwards-compatibility.

Its not clear to me why these couldn't be replaced by functions? Which code are you trying to maintain backwards compatibility with - users or internally?

Hey @Padarn We only want to maintain backwards compatibility with the user API, so identifiers have already been replaced by functions. you can take a look at the main branch. E.g., the main identify_effect function.

Ohh that is much more clear now. Thanks for the clarification @amit-sharma.


feat:Add navigation for sdc gallery app fix#154

fixes #154
Changes made :
1. Converted to single activity architecture
While converting to single activity architecture had to rename QuestionnaireActivity to QuestionaireFragment however I got a warning on intellij saying QuestionnaireActivity will be shadowed by class QuestionnaireFragment and so renamed it to MyQuestionnaireFragment (if someone could suggest a better name that would be great).
2. Added Navigation Graph
3. Added Navigation Args
To add the Nav args plugin as mentioned in the link on the linked issue had to add a classpath to the project level build.gradle file.
So should I avoid using safeargs or should I just add the classpath to project level build.gradle file?

With #334 merged, please rebase/merge and run ./gradlew sA to format your XML files. Also see https://github.com/google/android-fhir/wiki/Developer's-Guide#kotlin-style.


Safari Screen-Reader Very Slow

Safari gets into ‘busy’ mode every time a switch is made into or out of the calendar or on change of dates, and it takes a good 11-12 seconds to get functional again. Is it possible to get addressed as Safari/VoiceOver combination is very prominent among accessible usage?
Steps to reproduce
https://www.apple.com/voiceover/info/guide/_1124.html
Browsers Affected

[ ] Chrome
[ ] Firefox
[x] Safari
[ ] Edge
[ ] IE 11
[ ] iOS Safari
[ ] Android Chrome

Seems to be working fine with the latest version. Reopen if still relevant.


Tablet mode operator modifiers

The operation modifiers for tablet mode operation also appear in desktop mode but have no effect. They only work in tablet mode. We should make them invisible in desktop mode.
Also, is there a way to switch between tablet and desktop mode (e.g., if the browser is detected as tablet but the user has a keyboard connected)?

The operation modifiers for tablet mode operation also appear in desktop mode but have no effect. They only work in tablet mode. We should make them invisible in desktop mode.

The detection is based on whether the device has a touch capable screen. Platform.has.touch. And the events for mouse and touch are different, hence those mode operations have no effect using mouse.

Also, is there a way to switch between tablet and desktop mode (e.g., if the browser is detected as tablet but the user has a keyboard connected)?

[ ] Add a new option touch to preferences, with default Platform.has.touch.
[ ] Add the touch to URL Parameters.
[ ] Add a popup when using mouse in touch mode and touching in mouse mode.

Ah, okay. I think this is because my laptop also has a touch screen. On my desktop the modifiers are not visible. No need to fix (but maybe add a comment on the help page that modifier switches will show for devices with touchscreens).


A bug in Illuminate\Foundation\Http\FormRequest

Laravel Version: 5.8.16
PHP Version: 7.3.3
Database Driver & Version:

Description:
In this method:
protected function createDefaultValidator(ValidationFactory $factory)
{
return $factory->make(
$this->validationData(), $this->container->call([$this, 'rules']),
$this->messages(),$this->attributes()
);
}
It should be:
protected function createDefaultValidator(ValidationFactory $factory)
{
return $factory->make(
$this->validationData(), $this->container->call([$this, 'rules']),
$this->attributes(), $this->messages()
);
}
Or the messages and attributes method could not work
Steps To Reproduce:

Oh, I read a wrong demo, and made a mistake.
There is no bug


[SPARK-37101][CORE] In class ShuffleBlockPusher, use config instead of key

What changes were proposed in this pull request?
In class ShuffleBlockPusher, We can use config.REDUCER_MAX_SIZE_IN_FLIGHT and config.REDUCER_MAX_REQS_IN_FLIGHT instead of "spark.reducer.maxSizeInFlight" and "spark.reducer.maxReqsInFlight"
Why are the changes needed?
Does this PR introduce any user-facing change?
How was this patch tested?

Please fill the PR description as guided in the template.


Fix usage of editor.action.wordHighlight.trigger

Fixes: #225065

the ci is stuck; i'll reopen the pr


Added context support

Added context support for the As this is much-required feature for the golang Apps.

@harnash you can fork with this repo. we are also using context for opentracing. i have added ctx support in it.
https://github.com/tokopedia/osin

👍


Release tracking PR: base58ck v0.1.0

In preparation for release add a minimal changelog to the base58ck crate.
This crate is currently unreleased and has the version number correctly set to v0.1.0 - as of today, the crate name base58ck is available on crates.io.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

14 of 113 (12.39%)  changed or added relevant lines in 7 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.2%) to 83.919%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

base58/src/lib.rs
9
11
81.82%

bitcoin/src/address/mod.rs
0
3
0.0%

bitcoin/src/bip32.rs
0
10
0.0%

bitcoin/src/address/error.rs
0
24
0.0%

bitcoin/src/crypto/key.rs
0
25
0.0%

base58/src/error.rs
4
39
10.26%

Files with Coverage Reduction
New Missed Lines
%

bitcoin/src/crypto/key.rs
1
67.35%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
19674

Relevant Lines:
23444

💛  - Coveralls

Sadly we cannot publish this without first publishing a new release of internals with the impl_from_infallible macro in it (and then probably bumping the version of internals in all the other Cargo.tomls)

Bollocks, my bad.

We should probably put #2337 in first since it adds a single line to internals (though I think it can also be omitted because its just a compiler attribute)

I just realized that I can run cargo publish --dry-run even though I don't have publish rights. I'll try do that from now on release tracking PRs to catch errors like this.

I think that our allows have no effect on downstream users (clippy is kind enough not to lint dependencies) so it doesn't matter if we add them before or after a release.

Tagged and published 499f36f97264d62d59ac5177536ca643cf983bf2


Adding vertical scroll

Added new prop horizontalScroll, for changing scroll type in the card form

Please, bring your pull requests to my repo. I Will maintain this lib.
https://github.com/ebellumat/react-native-credit-card-input


MM-22487 Fix race condition causing the user to logout

Summary
On login when the channel screen mounted if the current channel wasn't yet selected there was a call to get the channel stats that will return a 401 from the server causing the user to force logout.
This was more prominent when using SSO as login method
Ticket Link
https://mattermost.atlassian.net/browse/MM-22487

@enahum
Error trying doing the automated Cherry picking. Please do this manually
+++ Updating remotes...
Fetching upstream
Fetching origin
+++ Updating remotes done...
+++ Creating local branch automated-cherry-pick-of-#3916-upstream-release-1.28-1581539170
Branch 'automated-cherry-pick-of-#3916-upstream-release-1.28-1581539170' set up to track remote branch 'release-1.28' from 'upstream'.
+++ Downloading patch to /tmp/3916.patch (in case you need to do this again)

+++ About to attempt cherry pick of PR. To reattempt:
  $ git am -3 /tmp/3916.patch

Applying: MM-22487 Fix race condition causing the user to logout
Using index info to reconstruct a base tree...
M	package-lock.json
M	package.json
Falling back to patching base and 3-way merge...
Auto-merging package.json
CONFLICT (content): Merge conflict in package.json
Auto-merging package-lock.json
CONFLICT (content): Merge conflict in package-lock.json
Patch failed at 0001 MM-22487 Fix race condition causing the user to logout
Use 'git am --show-current-patch' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

+++ Conflicts detected:

UU package-lock.json
UU package.json

+++ Aborting in-progress git am.

+++ Returning you to the master branch and cleaning up.


[CI] Fix for jsdoc hook in pre-commit

Info
This is a fix for jsdoc hook from https://github.com/godotengine/godot/pull/91597
Currently, the hooks in the pre-commit are only triggered if the specified files have been modified. In the case of the jsdoc hook, engine.js, config.js and features.js from the platform/web/js/engine directory are checked.
The problem is that jsdoc requires all these 3 files in the parameter line and not, as pre-commit does, only those that were modified in the commit.
When, for example, only feature.js is changed in a commit, an error will appear:
jsdoc....................................................................Failed
- hook id: jsdoc
- exit code: 1

[...]

Undefined symbol! Engine
/godot/platform/web/js/jsdoc2rst/publish.js:315
				throw new Error('Undefined symbol!');
				^

Error: Undefined symbol!

The solution is to permanently add the names of all 3 files to the argument line and disable the automatic addition of modified file names by pre-commit (pass_filenames: false).
        args:
          - --template
          - platform/web/js/jsdoc2rst/
          - platform/web/js/engine/engine.js
          - platform/web/js/engine/config.js
          - platform/web/js/engine/features.js
          - --destination
          - ''
          - -d
          - dry-run
        pass_filenames: false

Thanks!


Specify default Procfile through environment variable?

Would be nice if I could add this to my .zshrc and foreman would just always use Procfile.dev without having to always specify it as an argument:
PROCFILE=Procfile.dev

More generally, would it be possible to make foreman behave differently depending on a FOREMAN_ENV variable? (e.g. FOREMAN_ENV=staging should use Procfile.staging and .staging.env)

Why not just make it a function?
foreman_with_env() {
  foreman "$@" $(if [ -n "$PROCFILE" ]; then echo "-f=$PROCFILE"; fi)
}
alias foreman=foreman_with_env

Yep that would work... My bash skills are limited so thanks for the script :+1: .

No problem. After thinking about this more however, I agree with your original suggestion; being able to use environment variables as default behavior would be nicer. I'll poke around the code a bit later and if I can add the feature, I'll submit a PR.

Has this feature been added?


Model updates for Zoom integration

Description
Please include a summary of the change and which issue is fixed. Please also
include relevant motivation and context.
Fixes FP-3673
Checklist

[ ] The changes have been tested to be backwards compatible.
[ ] The OpenAPI schema and generated client have been updated.
[ ] New models module has been added to api generator xtask
[ ] New types/fields are well-documented and annotated.
[ ] The CHANGELOG is updated.

Please link any related PRs in other repositories that depend on this:

After merging, please merge related PRs ASAP, so others don't get blocked.

Keeping this in draft for a few weeks until we free up some time to figure out what we wanna do with it


Only test debug NaN printing on x86_64

NaNs are mangled in various ways on various platforms.  Some
of the fun possibilities:

sign bits sometimes can be cleared
payloads get zeroed
NaNs get forced to quiet
Merely passing a value into a function can trigger these behaviors
Trivial changes to the generated assembly can change this

This makes it very hard to validate the printing of NaN
details, since we can't be certain what value the formatter
actually saw.
So I'm simply disabling the checks for debugDescription on
NaNs except on x86_64, which has pretty well-understood NaN
support.
This should fix a test failure recently seen on 64-bit ARM hardware.
Note: Eventually, it might be nice to verify that the
debugDescription output always matches the expected
format:
   -?s?nan(\(0x[0-9a-f]+\))?

But it's not clear that anything stronger is really feasible.

@swift-ci Please test

I just added (most of) a check for the general format.

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 62132fda2c46f2e2f357154565ab2d201e38e654

Build failed
Swift Test Linux Platform
Git Sha - 62132fda2c46f2e2f357154565ab2d201e38e654


Problem to connect

I have problem to Connect to PLC with TC2 from Windows 11
I get this error msg.
11 Nov 14:25:55 - [warn] [ads-client-connection:ab52d61c2fbd59fa] Failed to connect <IP_ADDRESS>.1.1:801 at startup: ClientException: Connection to <IP_ADDRESS>:48898 failed (socket error -4077) (read ECONNRESET)
Ads Router is OK i have another program communicating ADS with the PLC and it works.
Version on Node and Node-Red
11 Nov 14:25:54 - [info] Node-RED version: v4.0.5
11 Nov 14:25:54 - [info] Node.js  version: v22.11.0
Version on node-red-contrib-ads-client is 1.4.0

I tried to set all connetion settings manualy and then it works


[WIP] spike showing failure notifier with options

The idea would be that normally you'd just set the failure notifier to something like:
RSpec::Support.failure_notifier = proc { |failure| method(:raise)(failure) }

but that aggregate_failures would change to something like:
def call(failure, options = {})
  failure.set_backtrace(caller) unless failure.backtrace
  failures << failure unless options[:eager]
end

^^ @myronmarston

Closing in favor of #210.  Figured you're focused on the rspec-core profiling issue so I'd move this forward.


Hide software keyboard

This seems like something test-butler should take care of?

adb shell settings put secure show_ime_with_hard_keyboard 0
http://www.drewhannay.com/2016/04/how-to-hide-android-soft-keyboard-in.html

Seems reasonable. I think we also want it in our emulator startup script still, cause it's annoying for devs to have to change it manually.

Is UiAutomation.html#executeShellCommand the right approach for setting this via on device Java code?
The issue with a startup script is it's not going to work on test clouds where the emulator isn't subject to user control.

Yeah, the startup script comment was just in reference to the blog post. Totally makes sense to add it in Test Butler too.
I don't think we'd need executeShellCommand. I think it would be [Settings.Secure#putInt](https://developer.android.com/reference/android/provider/Settings.Secure.html#putInt(android.content.ContentResolver, java.lang.String, int))

Is there a startup script somewhere? That'd be cool to look at.
Thanks for the pointer to Settings.Secure#putInt

It's on the list of projects to open source, but not quite ready yet 😄


check whether pods should be managed when ns is managed

What happened:
When I read the code, I found that for newly managed ns, all pods under the ns were directly managed without necessary checks.
For example, pods containing the label "istio.io/dataplane-mode: none" should not be managed.
ref: https://github.com/kmesh-net/kmesh/blob/main/pkg/controller/manage/manage_controller.go#L260
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

Kmesh version:
Others:

/assign


Upgrade netty to 4.1.22

Upgraded netty to 4.1.22

Thanks for the PR @sbouchex
Andrea

@andsel Any plan to release a new version with this upgrade ?

At the moment no, I've other fixes to do, and I want to group them all
together.
Andrea
On Thu, Jun 21, 2018 at 4:39 PM leplan73<EMAIL_ADDRESS>wrote:

@andsel https://github.com/andsel Any plan to release a new version
with this upgrade ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/andsel/moquette/pull/413#issuecomment-399126880, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA_Y0R6zYaLS6qKA8S9arFD8vweaYKKzks5t-6_qgaJpZM4Utit1
.


HoloLens 2 Tutorials TOC update

Updated TOC and included PC Holographic remoting tutorials and Azure Cloud tutorials as part of HoloLens 2 Tutorials

Fixed Typos

Docs Build status updates of commit ca0fdc4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/TOC.md
:white_check_mark:Succeeded
View

mixed-reality-docs/mr-learning-pc-holographic-remoting-01.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit ca0fdc4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/TOC.md
:white_check_mark:Succeeded
View

mixed-reality-docs/mr-learning-pc-holographic-remoting-01.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Giving "infores" some love?

Playing with your ROBOKOP instance, I noticed two things and would like to suggest some improvements for increased provenance transparency.
Use `primary_knowledge_source` only ever for the very original source
Lets look at a query like this:

MATCH (n:`biolink:Gene` {id: 'NCBIGene:5979'})-[r]-(d:`biolink:Disease` {id: 'DOID:0050430'})
RETURN n,r,d LIMIT 25

For the exact same association, you get a wild mix of primary_knowledge_source and aggregator_knowledge_source. I believe, while it is hard, that for aggregation of scientific evidence it is critical that the very original source making that statement is  primary_knowledge_source, and all other downstream sources should only ever be mentioned as aggregator_knowledge_source. In particular, "infores:monarchinitiative" should (at least if I am not mistaken, given infores:hpo-annotations", cc @kevinschaper?) never appear as a primary_knowledge_source. Same as pharos:

Neither should, and this is IMO very important, infores:ubergraph. Here it is critical that every single integrated edge gets infores:sourceontology (e.g. infores:uberon) for maximum transparency. Again, it would be great if the edge could point somehow to a version of the knowledge source (asserted_id: [ubergraph2023-01-01, pharos2024-03-4, monarch2024-03-04], etc), but I don't know how that's done in Biolink. This is not just about provenance. This is also about attribution: We want to make sure that when we deliver high Impact KGs like ROBOKOP to the science world, everyone know that "wow, uberon really made a difference to beef up the context for our node embeddings". This is only possible if we add that info on every single edge.

Either never or always aggregate "aggregator_knowledge_source"
Right now we have a mix of cases, like in the query above. The advantage of "always aggregate" is you can see immediately how well an edge is supported in the graph (how many aggregators have deemed it trustworthy). On the other hand, there is a risk of not being able to adequately integrate association metadata if it diverges across resources. I don't know the right answer to this, but in order to recommend preprocessing for ML tools (should the number of edges between two nodes matter?) I believe this has to be done consistently.

Otherwise looks great! You have 42 knowledge sources, and all of them appear in the infores registry, which is awesome!
These were my two cents!
cc @marcello-deluca

Thanks @matentzn, it's great to have more eyes on this.
For point one, that is certainly the intention. For the query you shared, I'm not sure those are all the exact same association. In fact, the reason there are so many edges that look similar is because they came from different primary knowledge sources, even when they came through the same aggregator. The edge merging algorithm in ORION uses the primary knowledge source as part of the criteria for determining whether two edges are the same, so edges with different primary knowledge sources are always kept separate.
uniprot -> pharos -> robokop
monarchinitiative -> pharos -> robokop
eram -> pharos -> robokop
ctd -> pharos -> robokop
This means that pharos has something about this kind of relationship from all of these underlying sources, as separate database entries. However, it appears we may have an issue with pharos because it is an aggregator of aggregators. Monarchinitiative is used because pharos has "Monarch" as the source database, and unfortunately, I don't think pharos provides the true primary source for those. Same for edges from CTD -> pharos.
We do have pharos as the primary knowledge source for some edges, but only when the real primary source could not be determined or for edge cases we never handled, but they should be considered mistakes or TBD (of course it is still helpful to identify and correct these cases).
For ubergraph, let's loop in @balhoff, but I'm under the impression that many (most?) of the edges from Ubergraph actually are generated by Ubergraph in a way where it should be considered the primary source. It is not simply aggregating knowledge from other sources, but generating edges using techniques like logical entailment.
Re: including versions of sources, that's definitely a good idea. ORION tracks source versions in graph metadata as best as it can (many sources do not provide real version identifiers), but does not include any of that inside the graph.
For your second point, I'm not sure I understand. We ingest some sources that are the primary knowledge source for their data, and some sources that are aggregators already. So to properly track that we necessarily have some edges with an aggregator knowledge source and some without. Maybe I'm misunderstanding what you mean though.

Thanks @matentzn, it's great to have more eyes on this.

Always happy to, just hope I am not a burden here!

For point one, that is certainly the intention. For the query you shared, I'm not sure those are all the exact same association.

Sorry for point one, I shared the picture not because it has two associations, but because it has primary_knowledge_source infores:pharos, which seemed unlikely (its possible though!). I understand that this is difficult, but when ingesting sources we should make sure that the provenance of every single edge we are importing can be traced to the scientific event that produced it (e.g., the group that extracted the information from some scientific paper).

In fact, the reason there are so many edges that look similar is because they came from different primary knowledge sources, even when they came through the same aggregator.

OK, got it. The "look similar" you say; of subject, relation, object is identical, there can still be biologically meaningful differences?

The edge merging algorithm in ORION uses the primary knowledge source as part of the criteria for determining whether two edges are the same, so edges with different primary knowledge sources are always kept separate.

Yeah, this makes sense. Thank you for clarifying. So when you have two relationships with the same primary knowledge source and different aggregators, you always merge?

Monarchinitiative is used because pharos has "Monarch" as the source database, and unfortunately, I don't think pharos provides the true primary source for those. Same for edges from CTD -> pharos.
We do have pharos as the primary knowledge source for some edges, but only when the real primary source could not be determined or for edge cases we never handled, but they should be considered mistakes or TBD (of course it is still helpful to identify and correct these cases).

If this information is not available, our great leaders like @balhoff and @cbizon should be encouraged to reach out and push the resources to include this provenance - not having it severely diminishes the credibility of our KGs.

For ubergraph, let's loop in @balhoff, but I'm under the impression that many (most?) of the edges from Ubergraph actually are generated by Ubergraph in a way where it should be considered the primary source. It is not simply aggregating knowledge from other sources, but generating edges using techniques like logical entailment.

Agreed thanks for pointing that out! But I don't think I saw infores:mondo anywhere in ROBOKOP.

Re: including versions of sources, that's definitely a good idea. ORION tracks source versions in graph metadata as best as it can (many sources do not provide real version identifiers), but does not include any of that inside the graph.

Awesome.

For your second point, I'm not sure I understand. We ingest some sources that are the primary knowledge source for their data, and some sources that are aggregators already. So to properly track that we necessarily have some edges with an aggregator knowledge source and some without. Maybe I'm misunderstanding what you mean though.

I think you commented the right things above. If this assumption is correct, the second concern is mitigated:

If two associations have diverging primary knowledge providers, they are both asserted as separate relations (I understand that's the case)
If two association from different aggregators have the same primary knowledge provider, they are merged into one association athe the knowledge aggregator metadata field is aggregated (i.e. both aggregators are included).

@EvanDietzMorris thanks so much, this is great feedback!

@matentzn it would be hard/slow to check if an Ubergraph edge is provided by an ontology (compared to just downloading edges). But it would be possible. I think a decent assumption would be to credit whichever ontology is the source of the subject term—what do you think about that?

Credit by subject is a good proxy if we don't have anything else, but my gut feeling tells me that in the long run, we should attribute edges directly, using rdfs:isDefinedBy during ubergraph construction, so you can attribute all the amazing RG stuff that ubergraph does (and @EvanDietzMorris rightfully points out!) to infores:ubergraph!
If we don't have a quick function in ROBOT "annotate-all-axioms" (I think we do in "extract"?) we can write one and inject as a robot plugin.


Force the gigasecond exercise to use a immutable version of Datetime

The current solution in the example has side effects.
function from(\DateTime $from)
{
    $interval = new DateInterval("PT1000000000S");
    return $from->add($interval);
}

The add method not only returns a new DateTime but it also mutates $from.
Given that I would call this function twice with the same value.
$date = new DateTime('1990-01-01');
from($date); // $date is now 2021-09-09
from($date); // $date is now 2053-05-18

This is clearly bad practice and is easily solved by using DateTimeImmutable.

Yes, good catch. The build is failing. Would you mind taking a look at it?
@Dog do you see any gotchas with this?

@kytrinyx It fails with the following error:
running tests for: bowling
This version of PHPUnit requires PHP 5.6; using the latest version of PHP is highly recommended.
make[1]: *** [test-assignment] Error 1
make: *** [test] Error 1

The travis build uses php 5.5 so its a compatibility issue and not an issue with the tests themselves.
A solution would be to update the .travis.yml to use php 5.6

I feel it really depends on what you are going to use gigasecond for, and the intent isn't really given from the readme.
Functionally, I see nothing wrong with this:
$date = new DateTime('1990-01-01');
from($date); // $date is now 2021-09-09
from($date); // $date is now 2053-05-18

It could be argued that you don't want to modify the original date time, but thats really left up to the implementor and the specification. I definitely agree the solution has a problem, but the issue isn't inherently its functionality. I think the break down was I assumed add would return a new Datetime without touching the old one - similar to the DateTime library in Python.
The only thing that bothers me about this solution, is I don't like the assumption you will be passed an immutable date time. What I would prefer is a solution that created a new DateTime object without modifying the original - that is regardless of if its input is immutable. We could then add in a test to check the original date time wasn't modified.

I like the direction of not constraining the solution space directly, but adding a test to check for immutability.

@Dog The mutable behaviour looks too much like an output argument to me and that didn't sit well. I modified the example to match your requirements.
Thank you for taking the time to respond!

This can probably be tried again on Travis by someone with permissions, and then merged.

I'll restart the build. Thanks, @petemcfarlane


Re-enable faster github CI runners

new test runners are working again.

Codecov Report

Merging #2732 (34b3583) into main (6d5bc2a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2732   +/-   ##
=======================================
  Coverage   81.21%   81.21%
=======================================
  Files          53       53
  Lines        5605     5605
=======================================
  Hits         4552     4552
  Misses       1053     1053

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Fix Issue 20523 - std.zip: Invalid data yields RangeError in ZipArchive constructor

This PR does two things: Moving from uint to ulong for support of files larger then 4GB. And adding access of the records via structs. It was easier to add both at once, because else I would have had to add hundreds of casts just to remove them again in the next PR.
This PR was allready planed for a while and I recognized that it fixes issue 20523 (and several similar ones) for free, because the calls to removeSegment are now done before accessing the data and therefore prevent the access of not existing data.

This is an old PR of mine, which I'm not going to handle in the near future, so I'm closing it - it's still accessible from the issue and I dislike having stalled PRs hanging around... Maybe there will be a time, when I'm coming back on std.zip.


Upgrade Simulation Chart to amCharts 5

Waiting on features to be added to amCharts 5:

[x] HTML Tooltips

mostly done in feature/amCharts5-upgrade Branch, but branch not up-to-date with latest SimulationChart Changes.

closes #68


AttributeError: 'Agent' object has no attribute 'request_login'

Hi @kevinpiac @aurelien-brabant , is the login feature implemented for python sdk?
I am trying to implement something similar to your chat-with-rust-book example and would like to enforce login
I get AttributeError: 'Agent' object has no attribute 'request_login' when tried with below code snippet
# in main
ferris = agentlabs.agent(env.agent_id)

# in handle_chat_message
ferris.request_login(
            conversation_id=message.conversation_id,
            text="Please login to access this feature"
            )

Hi @hitz02!
Thanks for reporting this. It's right, the documentation is a bit ahead of our codebase. The Python SDK currently does not support this feature.
Good news though, we're going to ship it today or tomorrow :)
I will keep you updated here!

Thanks for the update @kevinpiac, any update on paywall feature to monetize agents?

Thanks for releasing a new update, and I've DM'ed you on Discord

Also, can you share a simple e2e example on how to use request_login ?


[$500] IOS -  The Chat report displays the wrong Latest Sender's name in LHN after logging in

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:
PRECONDITION: The user has joined a public group chat with a long chat history. For Example:
- Bob-testing: https://staging.new.expensify.com/r/5593084223054221

Log in to the account that has joined a public/group chat with a long chat history.
On the LHN: check the name of the Latest Sender displayed.
Open the chat to view chat's details.
Verify the actual sender who sent the Last Message in the chat.
Compare the Sender's name from step #4 with the name obtained in step #2.
Observe that the latest sender's name is different.

Expected Result:
The latest sender's name should match between step #4 and #2
Actual Result:
The latest sender's name is different.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[x] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/93399543/8b182771-c6c3-41bb-8148-87a2c048ebc2

Expensify/Expensify Issue URL:
Issue reported by: @tranvantoan-qn
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1693213921747209
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01bd85da703f1b3f23
Upwork Job ID:<PHONE_NUMBER>104852480
Last Price Increase: 2023-09-26

Applying the external label

Still waiting on proposals.
Assigning a new BZ member, I am out on sabbatical

Not overdue; waiting on proposals!

Still waiting for proposals!

Still waiting for proposals.

For more information, there are instances where the issue has been reported:

https://expensify.slack.com/archives/C049HHMV9SM/p1687647652432809
https://github.com/Expensify/App/issues/21510#issue-1773237327

Thanks! @ArekChr can you still reproduce this issue?

@isabelastisser
Yes, it's still happening, please look at the attached image:

Thanks! Waiting for proposals.

Issue not reproducible during KI retests. (First week)

Not overdue.

Issue not reproducible during KI retests. (Second week)

Don't know how you reproduced it, but it is still happening as always
The issue will happen AFTER logging in and accessing the chat room first time, so we need:

Logout the app
Login again
Check the last sender's name showing on the chat
Click on the chat to view details, and compare the name with #3.

Yes. It is still happening as I explained here
https://github.com/Expensify/App/issues/26609#issuecomment-1747975327

I am Michał from Callstack - expert contributor group. I’d like to work on this job.

@tranvantoan-qn
Hey, I've managed to reproduce something similar but I'm not sure what the correct outcome here should be as the scenario is a bit more complex.
https://github.com/Expensify/App/assets/23487612/5cac4240-6a48-4233-84d0-a5395a51a0bb
After logging in, as the latest message in the chat, I see last sender from the message that is only visible to me and message text from the chat that was last message visible to everyone participating. What should be the proper behaviour here?
Should messages only visible to me be ignored and not displayed as the last message and we should display latest message visible to everyone with the correct sender OR we should display message only visible to me with the correct sender?

@MrMuzyk Mr
I don't know if we need to show the "only visible to you" message on the main page (on Desktop, we call it LHN - Left Hand Navigator) - as this should be confirmed by Expensify's team member
But in my opinion, there is no problem if we show it as long as the sender's name and the message is the same between the main page and the details page. So in this case:

For you: It would show Bob Ladue: testing 1 2 3; can I....
For me (and maybe others who didn't get the message  "only visible to you"): It would show<EMAIL_ADDRESS>2

Issue not reproducible during KI retests. (Third week)

@MrMuzyk @mvtglobally can this still be reproduced? Should we close it?

@isabelastisser I believe it is still reproducible. Gonna post my proposal below

Proposal
Please re-state the problem that we are trying to solve in this issue.
When user logs in and looks at the report on the LFH no Latest Sender is displayed or Latest Sender that is displayed is incorrect. Once user clicks and opens the report then additional info is fetched and from now on till logout Latest Sender will be displayed correctly for such report.
What is the root cause of that problem?
Latest Sender detailed info is not provided together with the report details. In ONYX under report_${reportID} entry the only relevant data to display Lastest Sender with the last message is lastMessageText. lastActorAccountID  is also provided but it’s not enough to display the name. In codebase inside SidebarUtils.js lib there is an attempt to retrieve display name for Latest Sender - https://github.com/Expensify/App/blob/main/src/libs/SidebarUtils.js#L321
In majority of our cases it’s empty as there are no details for the ID provided in report object (lastActorAccountID) and sometimes it comes incorrect due to the fact that sometimes messages visible only to logged in user are treated as latest message and sometimes not.
What changes do you think we should make in order to solve the problem?

I believe a proper solution to this issues would be to provide another field in the report object saved in ONYX under report_${reportID}. Apart from lastMessageText new field should be added, for example: lastActorDisplayName, that would contain the display name of Latest Sender. This would allow us to consistently and correctly display the name of the Latest Sender.
What alternative solutions did you explore?
I was looking for a way to fetch the missing data myself as ID of the user we want to get the display name for is provided but I didn’t succeed. I couldn’t find a method that would allow me to fetch detailed data of the user based on the provided ID. On top of that this wouldn’t be recommended way to fix that issue because it would require introducing another API call as missing details are not present in ONYX at this point and would have to be fetched.

thanks @MrMuzyk ! @aimane-chnaif can you please review it? Thanks!

Bump @aimane-chnaif

I haven't seen the issue like that, and the public rooms I've joined are:

Scaling New Heights 2023 - #activityfeed: https://staging.new.expensify.com/r/5303403427249671
Scaling New Heights 2023 - #bobs-testing:  https://new.expensify.com/r/5593084223054221

Yes, I tried with completely new account.

Can you see the joined room on account B?
If you logged out on B and re-login, can you still see that room?

Hey, I’ve started work on some wave6 features that are high-priority and someone else from Callstack will take over this issue

@kbecciv @aimane-chnaif
According to this conversation I've missed
Can you help me to remove the link https://staging.new.expensify.com/r/5303403427249671 from the  Issue description
@MrMuzyk Please note that we don't need to send a new message to the rooms in order to reproduce the issue.

Hey @tranvantoan-qn , I've removed the link!
@MrMuzyk, can you please assign the new contributor who will replace you?

@isabelastisser Hi! I'm Bartek from Callstack and I want to take over this issue, please assign me :)

hey @barttom, any updates?

I agree with the proposal from @MrMuzyk. Looks like we waiting for reviewing proposal by @aimane-chnaif.

Ah interesting, so you might not have the personal details of the last actor accountID and so it just falls back to the most recent visible action?
If we added the key lastActorDisplayName, that would essentially be the fallback if don't have that accountID in your personal details right because otherwise it would pull the actual display name or email.
I don't really like the idea of adding another key to determine which name to show considering it's only applicable for certain users, and then there's always the question of which display name to use - from the lastReportAction or the report and keeping those in sync we'd probably have to make a decent amount of onyx optimistic updates
I'm leaning towards the second solution since that seems to be how we handle these other scenarios where we don't have personal details for a user.
Though curious if ya'll agree (@puneetlath / @Beamanator)

Ah interesting. So we get the personalDetails in the openReport command, but since that hasn't been called yet it can be wrong. I agree with just not showing a display name in the case where we don't have it.

Yeah, we'll need to handle that on the back-end.

Offers sent in Upwork. All set!


[Feature][DISCUSS] operator support customize some feature configurations,like parallel

Search before asking

[X] I had searched in the feature and found no similar feature requirement.

Description
Sometimes transform need more parallel, so i think operator support customize some feature configurations.
Usage Scenario
No response
Related issues
No response
Are you willing to submit a PR?

[x] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

+1
Yes, It a good idea.

Do we set each operator individually or set the  parallelism uniformly?  @ashulin

@liugddx My opinion is the same as the [spark scenario] #2965, the user can set the parallelism of a single source and sink, and can also set the global parallelism. When the parallelism of the plugin does not exist, the global parallelism is used.

@liugddx My opinion is the same as the [spark scenario] #2965, the user can set the parallelism of a single source and sink, and can also set the global parallelism. When the parallelism of the plugin does not exist, the global parallelism is used.

I found that some operators are specified in parallelism itself.like jdbcSource and jdbcSink.Should we have uniform Settings? like this pr.

@liugddx I would like to have uniform the parallelism option Settings


Move RuleRating struct from aggregator to utils in order to be used by several projects

Description
Tule RuleRating struct will be used by Aggregator and Smart Proxy, so it needs its place in the types package
Type of change

Refactor (refactoring code, removing useless files)

Testing steps
Regular CI

Codecov Report

Merging #191 (8a9caec) into master (05133bf) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #191   +/-   ##
=======================================
  Coverage   90.54%   90.54%
=======================================
  Files          22       22
  Lines         825      825
=======================================
  Hits          747      747
  Misses         64       64
  Partials       14       14

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 05133bf...8a9caec. Read the comment docs.