When a user is offline clicking on internal links shows a growl that you are offline - Reported by @parasharrajat

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Send a message which contains https://www.expensify.com/inbox.
Click on the link.

Expected Result:
A growl is shown to the user that you are offline.
Actual Result:
User should be taken to that link as it is a valid url and not of our app's domain.
Workaround:
unknown
Platform:

Where is this issue occurring?

Web

Version Number: 1.1.91-0
Reproducible in staging?: Y
Reproducible in production?: Y
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/43995119/187115018-c45f595d-ce40-48b6-a420-b4786fc11259.mp4
https://user-images.githubusercontent.com/43995119/187115081-5f418034-a4a0-4978-8371-c9316e362722.mp4
Expensify/Expensify Issue URL:
Issue reported by: @parasharrajat
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1660322955708219
View all open jobs on GitHub

Adding @AndrewGable cause you recently did some work on the links so you might know a bit more of the logic.
This is not a deploy blocker so I think it could be an external.
That said @mvtglobally the 2 videos seems to be the same, not sure if that is on purpose or one of them is the wrong, can you check please

I'm settign this one as an external

https://www.upwork.com/jobs/~01e669d3893dab99be

Hi all,
There are 2 related files to this issue:

AnchorRenderer.js
which is detecting that we are actually navigating to oldExpensifyPath
Link.js
which fails early if no network because it assumes that it needs authToken for the link to work. For the authToken exchange - network is required

Proposal:
Fallback to just navigating to the link without authToken(short-lived authentication token) param if no service, otherwise append authToken param. This will open the link in a new browser tab like any other link.
After network is online again everything seems to work fine - the app will even maintain user's login state if there was any, otherwise it will show login page.
Bonus:
Do we even need authToken exchange at all? From my testing the observable behavior of the app doesn't change with or without it.

PROPOSAL
Root Cause
There is a filter process in the AnchorRenderer to verify the links as internal links and external links. Issue is, this process thinks that expensify.com/index is an external link because it does not start with "www.".

As you can see, it checks for an URL if they start with "EXPENSIFY_URL" constant. And this constant if set to "https://www.expensify.com/'" right now.
While the dynamic links are loaded into this component at first, links without a "www." gets their href as "https://expensify/index". So when checking if they are an internal or external, since the link doesn't have "www." prefix, they are treated as an external link. And external links are handled with opening a new window as per in the code below.

Solution
We can change all link to handled without "www." prefix. To do that, we can delete this prefix if it exists.

And in the constant side

Now all links that has www. or not will be treated as same in the application.
Solution 2
We can define a new constant for Expensify url without a "www." prefix and change the filter accordingly.
Thanks!

@chiragsalian not sure why it assigned you if I can myself take this?
Feel free to unasign if you want, or if you want to keep it unasign me 🤷🏼

@mananjadhav please see above proposals!

Hey @arielgreen I am a bit tied up with my backlog. I didn't see this issue, can you please reassign this one.

not overdue, we had to assgin a new C+

@rushatgabhane see above proposals, please

@sobitneupane can you please review the proposals above?

Fallback to just navigating to the link without authToken(short-lived authentication token) param if no service, otherwise append authToken param. This will open the link in a new browser tab like any other link.

@vladamx I like your idea  here. But can you be more concrete with your proposal? How and where are you planning to "Fallback to just navigating to the link" ? Some code blocks will make it more clear.
@quidam93 I don't think your proposal is aligned to the problem description.

@sobitneupane
Either this

OR

Do we even need authToken exchange at all? From my testing the observable behavior of the app doesn't change with or without it.

@vladamx  We need authToken to signin user to the browser if he/she is not already logged in.
@sketchydroide @chiragsalian Proposal from @vladamx with explanation in https://github.com/Expensify/App/issues/10625#issuecomment-1246475018 and code in https://github.com/Expensify/App/issues/10625#issuecomment-1263852727 first image looks good to me.
🎀👀🎀 C+ reviewed

Proposal LGTM, i didnt realize i was still assigned here. Unassigning myself since you are working on the issue @sketchydroide 🙂
Since he's approved it as well assigning issue to @vladamx. Feel free to create the PR.

Applied to Upwork.
PR 5.10.2022

Offer sent

PR still under review

Issue not reproducible during KI retests. (First week)

Payment due tomorrow.

Paid.


Include wildcard field in comparison of challenge specs

What this PR does / why we need it:
Previously we could get into a situation where we 'miss' a challenge resource for an order if that order has *.example.com and example.com named on the same certificate.
This PR includes the 'Wildcard' field in the comparison that dictates whether to create a new challenge resource or not.
Release note:
NONE

/retest

/retest

/retest


Support for signing requests for AWS Elasticsearch

What kind an issue is this?

[x] Feature Request.

Issue description
If we want to fetch or write data to/from an AWS Elasticsearch setup, there is a need to sign requests as per the below documentation:
https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-ac.html#es-managedomains-signing-service-requests
As far as I understand, this is not supported by the "elasticsearch-hadoop" library as of now. If spark is running on AWS, one could configure IP address/host based IAM policies, but if data needs to be accessed from outside of AWS, there is not a good solution for this as far as I know.
The ES proxy (https://github.com/abutaha/aws-es-proxy) might be a possible way to get around this, but the problem there is that the infrastructure teams usually hesitate to provide tools that anybody can access without authenticating and dev teams don't really want to maintain things like this proxy.
Version Info
OS:         :  Centos 6.8
JVM         :  1.8
Hadoop/Spark: Spark 2
ES-Hadoop   :  6.2.4
ES          :  AWS ES 5.5

Feature description
See description above

Hi @prashs, thanks for opening an issue for this. We currently have an issue that covers this problem at #626. If you would like to add any information on to that issue as well as any +1's that would be great.
Since we have an issue open for this already, I'll go ahead and close this. Let me know if there's any reasons you feel like this should be re-opened and I'd be happy to take a look.


Make FruitHandle and MultiFruit public

Hi! I've been coding dynamic Collector creation in my project and figured out that it was impossible to store FruitHandle inside struct or Vec because of (in)visibility of FruitHandle and MultiFruit. Is it ok to make them visible?

Codecov Report

Merging #1360 (fdea97e) into main (944302a) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1360   +/-   ##
=======================================
  Coverage   94.28%   94.29%
=======================================
  Files         234      234
  Lines       42528    42528
=======================================
+ Hits        40099    40102    +3
+ Misses       2429     2426    -3

Impacted Files
Coverage Δ

src/collector/mod.rs
48.21% <ø> (ø)

src/collector/multi_collector.rs
99.30% <ø> (ø)

src/fastfield/serializer/mod.rs
92.75% <0.00%> (+1.44%)
:arrow_up:

src/directory/directory.rs
95.58% <0.00%> (+1.47%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 944302a...fdea97e. Read the comment docs.


Only cache payloads matching the current session

Goal

Testing

Release Notes

WHAT:
WHY:
WHO:

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#855  👈
#854
#846
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @bidetofevil and the rest of your teammates on  Graphite

Merge activity

May 17, 1:24 PM EDT: @bidetofevil merged this pull request with Graphite.


refactor: Add typed error enums for square and offset calculations

Rewrite some of the functions on the board to return dedicated Error types instead of returning plain &strs. This makes it possible to get better information as to exactly what went wrong when calling a function.

Appreciate the response! I also appreciate the really nice atomic commits in the PR.


Quaternion.GetEuler() and Quaternion(Vector3 eulerYXZ) not found in Godot 4.0.beta10.mono

Godot version
4.0.beta10.mono
System information
Windows 11
Issue description
When trying using the method Quaternion.GetEuler() or the constructor Quaternion(Vector3 eulerYXZ), I'm getting MissingMethodExceptions in the current Godot 4.0.beta.10.mono.
E 0:00:02:0034   Godot.NativeInterop.godot_bool Godot.Bridge.CSharpInstanceBridge.Call(IntPtr , Godot.NativeInterop.godot_string_name* , Godot.NativeInterop.godot_variant** , Int32 , Godot.NativeInterop.godot_variant_call_error* , Godot.NativeInterop.godot_variant* ): System.MissingMethodException: Method not found: 'Godot.Vector3 Godot.Quaternion.GetEuler()'.
  <C++ Error>    Exception
  <C++ Source>   :0 @ void root._Process(Double )()
  <Stack Trace>  :0 @ void root._Process(Double )()
                 Node.cs:1809 @ Boolean Godot.Node.InvokeGodotClassMethod(Godot.NativeInterop.godot_string_name& , Godot.NativeInterop.NativeVariantPtrArgs , Godot.NativeInterop.godot_variant& )()
                 CanvasItem.cs:1443 @ Boolean Godot.CanvasItem.InvokeGodotClassMethod(Godot.NativeInterop.godot_string_name& , Godot.NativeInterop.NativeVariantPtrArgs , Godot.NativeInterop.godot_variant& )()
                 Control.cs:3155 @ Boolean Godot.Control.InvokeGodotClassMethod(Godot.NativeInterop.godot_string_name& , Godot.NativeInterop.NativeVariantPtrArgs , Godot.NativeInterop.godot_variant& )()
                 root_ScriptMethods.generated.cs:38 @ Boolean root.InvokeGodotClassMethod(Godot.NativeInterop.godot_string_name& , Godot.NativeInterop.NativeVariantPtrArgs , Godot.NativeInterop.godot_variant& )()
                 CSharpInstanceBridge.cs:24 @ Godot.NativeInterop.godot_bool Godot.Bridge.CSharpInstanceBridge.Call(IntPtr , Godot.NativeInterop.godot_string_name* , Godot.NativeInterop.godot_variant** , Int32 , Godot.NativeInterop.godot_variant_call_error* , Godot.NativeInterop.godot_variant* )()

Steps to reproduce
A simple C# script should be enough to reproduce the error:
public partial class root : Control
{
  public override void _Process(double delta)
  {
    var q = Quaternion.Identity;
    var eu = q.GetEuler(); // <- throws C++ exception
  }
}

Minimal reproduction project
N/A

the constructor Quaternion(Vector3 eulerYXZ)

There's no such constructor, there's Quaternion.FromEuler(Vector3 eulerYXZ) static method instead:
https://github.com/godotengine/godot/blob/57267709e665f1683e79bd5d3432be2be5db6c1d/modules/mono/glue/GodotSharp/GodotSharp/Core/Quaternion.cs#L570-L576

A simple C# script should be enough to reproduce the error:
public partial class root : Control
{
  public override void _Process(double delta)
  {
    var q = Quaternion.Identity;
    var eu = q.GetEuler(); // <- throws C++ exception
  }
}

Can't reproduce, works just fine for me (v4.0.beta10.mono.official [d0398f62f]).

The MissingMethodException means that the method you are using could not be found, this is because the GodotSharp assembly referenced by the Game assembly and the one used on runtime are different.
One of the reasons why we ask for a minimal reproduction project is to check the .csproj file and other things that could be causing the issue. In this case, I would make sure the Project Sdk is the right version (should be Godot.NET.Sdk/4.0.0-beta.10).
Are you referencing other C# projects? Make sure they also reference the same version of the GodotSharp assembly; otherwise, the may be using methods that no longer exist.
In your case, the Quaternion.GetEuler() method does not exist, it was replaced with Quaternion.GetEuler(EulerOrder order = EulerOrder.Yxz), since the parameter is optional you may think it's the same method but the method signature is different which means this breaks binary compatibility so an older version of the GodotSharp assembly would reference a different method, causing the MissingMethodException.

Are you referencing other C# projects? Make sure they also reference the same version of the GodotSharp assembly; otherwise, the may be using methods that no longer exist.

That was exactly it! I have a multi-project setup and forgot to update the reference in one of the projects... silly me! Sorry for the false alarm, teaches me that I should always test in a minimal sample project first before opening an issue.


Remove epoch 12 user warning

Fix #360.

In the docs there are also prominent warnings like this one: https://www.vast-survey.org/vast-tools/components/query/, and is repeated on other pages.
All the per-page warnings could probably be removed now with perhaps just a warning on the data access page in getting started, would you agree?

Agreed!

Is it worth putting a note about it on the data page or just leave it to the changelog?
I was always uneasy at vast tools containing information about the actual data release as that should be its own thing.
But good to go in any case!

I think it's probably good as-is.
There's a warning in the data access section of the docs that states which version of the data release the package is compatible with (although that's probably outdated now). Maybe it's worth having something more explicit on the introduction page that lists what versions of the data and pipeline the version of tools works with. I'll add that to my to-do list in #352/#354.


Render Filter with customBodyRender

Expected Behavior
Render Filter-Values with customBodyRender

Current Behavior
If the column is a object the Filter shows [object Object] and not the customBodyRender return.
I have to try it with the custom filter option, but i do not get the filterdata to display the select options.
I think to a solution like that:
{filterData[index].map((filterValue, filterIndex) => (
              <MenuItem value={filterValue} key={filterIndex + 1}>
{column.customBodyRender ? (column.customBodyRender(filterValue)):
                (<React.Fragment>{filterValue != null ? filterValue.toString() : ''}</React.Fragment>)}
              </MenuItem>
            ))}

Your Environment

Tech
Version

Material-UI
4.5.1

MUI-datatables
2.12.1

React
16.11

Hello @svensq. I think I need more information about what you're trying to accomplish on my end. Can you demonstrate the bug, if there is one, in a codesandbox?

Sorry, I have found my mistake now. I return a Component by customBodyRender. Thank you.


compiler watchOptions are ignored in compiler.watch

I have a compiler with watch options:
var compiler = webpack({
    ...
    watchOptions: { ... }
    ...
});

then I start watching:
compiler.watch(watchOptions, function ( error, stats ) {
    // why I should put watchOptions here again?
    // they are already set for compiler as described in documentation
});

it would be nice to be able to just start watching as it has all the necessary info:
compiler.watch(function ( error, stats ) {
    ...
});

or even without callback as I may not need error/stats info:
compiler.watch();

You don't need the watchOptions in the configuration. You only need to provide it in the watch call.
watchOptions in configuration is only used by the CLI. Just like the devServer key which is only used by the dev server CLI.

@sokra quote from documentation:
"(only used when using CLI or simple node.js API)"
and I exactly use it in simple node.js API.
From architectural point of view maybe it's reasonable to split page configuration to a couple of config files.

yep thanks, I changed the documentation.

It is still confusing.
The whole page https://webpack.github.io/docs/configuration.html is clearly about the single configuration object passed to webpack.
It's rather strange to describe some options among compiler options but forbid to use these options for compiler.

This issue was moved to webpack/webpack.io#125


adds version protocol command, returns not implemented

Some clients use Memcached::getVersion to determine if the server is running and the connection is working.  The current implementation throws an error, which results in clients making the mistaken assumption the server is not responding.
This PR adds support for the version protocol command, but returns "not implemented". This allows the client to verify the connection is successful, but does not return any actual versioning information since we do not actually have a version number.
Usage
$ telnet <proxy-ip> <proxy-host>
Trying <proxy-ip>...
Connected to <proxy-ip>.
Escape character is '^]'.
version
VERSION not implemented

Alternatives
-VERSION appengine_proxy.v1
-VERSION 1
Using "not implemented" is the safest/least complicated solution, but either of these alternatives would also be okay.

Add a test?

LGTM

ok, added tests

I think this is good for now, but maybe we can figure out something better. How about just "App Engine"?
The spec doesn't say there needs to be a version number. My memcache on ubuntu a string in addition to the version number.
$ nc localhost 11211
version
VERSION 1.4.14 (Ubuntu)

LGTM


fix clean the tmp file of the default bucket

Codecov Report

Merging #2174 (cfa0f74) into main (7dd011a) will decrease coverage by 0.03%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #2174      +/-   ##
==========================================
- Coverage   61.77%   61.73%   -0.04%
==========================================
  Files         137      137
  Lines       24853    24850       -3
==========================================
- Hits        15353    15342      -11
- Misses       7726     7733       +7
- Partials     1774     1775       +1

Impacted Files
Coverage Δ

pkg/gateway/gateway.go
44.82% <0.00%> (+0.21%)
:arrow_up:

pkg/vfs/reader.go
77.18% <0.00%> (-2.56%)
:arrow_down:

pkg/vfs/writer.go
78.64% <0.00%> (-1.02%)
:arrow_down:

pkg/meta/sql.go
66.44% <0.00%> (ø)

pkg/meta/redis.go
71.23% <0.00%> (+0.04%)
:arrow_up:

pkg/chunk/disk_cache.go
80.33% <0.00%> (+0.48%)
:arrow_up:

pkg/chunk/utils_unix.go
93.33% <0.00%> (+20.00%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7dd011a...cfa0f74. Read the comment docs.


Run on Physionet

Hi,
Thank you for this awesome paper and great repository.
I wanted to try this model on MIMIC III and Physionet datasets and I see that there is a physionet.py file in the datasets folder.
If you already tried this on Physionet I would really appreciate it if you can share your code for running it with the configurations you used?
Thanks.

Hi Iman,
Thank you for your kind words. We have indeed. And CKConv seems to work very well as we'll:)
CKConv gets 0.895 Test AUC in Kidger et al 2020's setting.
Unfortunately, I am not close to my PC right now. I will make pretrained models as well commands to execute the code online first thing Monday. Does that sounds good?
Cheers,
David

Great that would be awesome.
I tried running it and ran into some problems so would be great to see your code.

Hi Iman,
I have now modified the repository. Now you can find the execution line used for the PhysioNet experiments, as well as a pretrained checkpoint. This checkpoint should give you a test AUC of 0.8965.
Please let us know if everything works good for you :)
Cheers,
David

Hi David,
I am getting a test AUC of 0.9310 with this model on Physionet which is higher than what you reported (0.8965).
How did you get that number? Is there anything that I'm missing?
Thanks,
Iman.


Update to support Allen NLP 1.0

This branch turned out to be the main functional changes from https://github.com/allenai/allennlp-semparse/pull/23
With these changes it is possible to run a wikitables prediction using AllenNLP 1.0.

Tests were not modified since I am not familiar with them.  Instead, the following simple script was used:
from allennlp.data.tokenizers import SpacyTokenizer
from allennlp.predictors import Predictor
import allennlp_models.rc
from allennlp_semparse import predictors, models
from allennlp_semparse.dataset_readers import wikitables

archive_url = "https://storage.googleapis.com/allennlp-public-models/wikitables-model-2020.02.10.tar.gz"
wikitables = Predictor.from_path(archive_url, "wikitables-parser")

data = { "question": "who has three apples?", "table": "Name\tApples\tOranges\nBill\tTwo\tFour\nTed\tThree\tFive" }
print(wikitables.predict_json(data))

Thanks, this is super helpful!  I'll take it from here, hopefully within the next day or two.

Thanks for getting this started @tedgoddard, it's now good to go.


UnsupportedSigner getSignedUrl

I recently changed from using aws-sdk and secret key credentials to this fork with apiKeyId and serviceInstanceId.  So far so good with the exception of getSignedUrl has started throwing the error below.  Do I need to change something to make it work with this type of authentication?  Thanks!
const s3 = new AWS.S3({"apiKeyId": "<>",
			endpoint: 's3-api.us-geo.objectstorage.softlayer.net',
			ibmAuthEndpoint: "https://iam.ng.bluemix.net/oidc/token",
			"serviceInstanceId": "<>"
		});
var params = {Bucket : BucketName,
			Key : ObjectKey,
			Expires: ExpirationTime};

		s3.getSignedUrl('getObject', params, function(err, data) {
			if (err) {
				console.log('copyObject Error',err, err.stack); // an error occurred
			} else {
				console.log('copyObject Returned',data);        // successful response
			}
		});

UnsupportedSigner: Presigning only supports S3 or SigV4 signing.
    at Request.signedUrlBuilder (/Users/Kyle/apex-designer-latest/node_modules/ibm-cos-sdk/lib/signers/presign.js:35:26)
    at Request.callListeners (/Users/Kyle/apex-designer-latest/node_modules/ibm-cos-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/Users/Kyle/apex-designer-latest/node_modules/ibm-cos-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/Users/Kyle/apex-designer-latest/node_modules/ibm-cos-sdk/lib/request.js:671:14)
    at Request.transition (/Users/Kyle/apex-designer-latest/node_modules/ibm-cos-sdk/lib/request.js:22:10)

Unfortunately presigned URLs need to have a public/private key pair to calculate a signature.  Soon the service credential will also provide an access/secret key pair that can be associated with an existing Service ID - this should be publicly available in the next couple of weeks.

I wanted to find out if there is an update to this issue regarding whether having an access/secret key pair is available, which would then would enable me to use the getSignedUrl method. Thanks!

@abe-m1 Bluemix can give you HMAC credentials now, which are the accesskeyid/secretaccesskey pair. If you use HMAC, be sure to also set signatureVersion: 'v4' when you provide the credentials.

I guess https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-presign-url also seems to suggest that HMAC is the only way to get a signed URL.

Yes, generating a presigned URL requires creating a client with HMAC
credentials.  In effect, a presigned URL is just v4 signature auth with the
value passed in a query parameter instead of a header.
We had looked into the possibilities around using scoped or single-use
tokens at one point, but at the time it wasn’t feasible. As as these
libraries continue to be so tightly coupled to the S3 API, I imagine
HMAC/v4 will remain the only way to generated a presigned URL.
On Tue, Mar 22, 2022 at 6:31 AM Matt Riedemann @.***>
wrote:

I just noticed #88 https://github.com/IBM/ibm-cos-sdk-js/issues/88 and
figured maybe that was my problem too, so I updated to the latest version
(1.11.0) but I'm still getting the error:
UnsupportedSigner: Presigning only supports S3 or SigV4 signing.\n at
Request.signedUrlBuilder
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/signers/presign.js:33:26)\n
at Request.callListeners
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/sequential_executor.js:106:20)\n
at Request.emit
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/sequential_executor.js:78:10)\n
at Request.emit
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/request.js:683:14)\n
at Request.transition
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/request.js:22:10)\n
at AcceptorStateMachine.runTo
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/state_machine.js:14:12)\n
at
/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/state_machine.js:26:10\n
at Request.
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/request.js:38:9)\n
at Request.
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/request.js:685:12)\n
at Request.callListeners
(/home/osboxes/ibmq/sw-ibmq-results/node_modules/ibm-cos-sdk/lib/sequential_executor.js:116:18)
I also tried setting signatureVersion: 'iam' but that doesn't help.
—
Reply to this email directly, view it on GitHub
https://github.com/IBM/ibm-cos-sdk-js/issues/11#issuecomment-1075183554,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADLL7REVLVPCZRJTUP3A33VBHDTNANCNFSM4EGJVVDQ
.
You are receiving this because you commented.Message ID:
@.***>

Yes, generating a presigned URL requires creating a client with HMAC credentials. In effect, a presigned URL is just v4 signature auth with the value passed in a query parameter instead of a header.

Thanks for quickly responding and confirming.


LLM explain

demo
MedAL

https://arc.net/l/quote/dizfzncf

'''
we have taken a basic biologic RPA to elucidate the pathophysiology of BPD bpd the CLD based on cellmolecular mechanisms of physiologic lung OD stretch coordinates PTHrP pthrp signaling between the ATII cell and the mesoderm to coordinately upregulate key genes for the homeostatic FB phenotype including peroxisome proliferator G1 TRG ppargamma adipocyte differentiation related protein adrp and OB and the retrograde stimulation of type ii cell surfactant synthesis by leptin each of these paracrine interactions requires cellspecific receptors on adjacent cells derived from the mesoderm or endoderm respectively to serially upregulate the signaling pathways between and within each celltype it is this PET compartmentation that is key to understanding how specific agonists and antagonists can predictably affect this mechanism of AM homeostasis using a wide variety of pathophysiologic insults associated with bpd barotrauma oxotrauma and infection we have found that there are type ii cell andor FB cellmolecular effects generated by these insults which can lead to the bpd phenotype we have exploited these cellspecific mechanisms to effectively prevent and treat lung injuries using ppargamma agonists to sustain this signaling pathway it is critically important to judiciously select physiologically and developmentally relevant interventions when treating the preterm neonate
''', what does PTHrP mean? please just tell me the answer don't explain it.
parathyroid hormonerelated protein

chatgpt 4o

Parathyroid hormonerelated protein
可以用Levenshtein距离
llama3
https://huggingface.co/spaces/ysharma/Chat_with_Meta_llama3_8b

Instruction Tuning for Large Language Models: A Survey
https://arxiv.org/pdf/2308.10792
FINETUNED LANGUAGE MODELS ARE ZERO-SHOT LEARNERS
https://arxiv.org/pdf/2109.01652

生物医学大语言模型时间轴
gantt
    title 生物医学大语言模型发展时间轴
    dateFormat  YYYY-MM-DD
    section 2019年
    BioBERT           :a1, 2019-01-01, 2019-12-31
    BlueBERT          :a2, 2019-01-01, 2019-12-31
    ClinicalBERT      :a3, 2019-01-01, 2019-12-31
    section 2020年
    PubMedBERT        :b1, 2020-01-01, 2020-12-31
    section 2021年
    BioMegatron       :c1, 2021-01-01, 2021-12-31

生物医学大语言模型时间轴
2019年
BioBERT: 基于BERT架构，专为生物医学领域设计，预训练数据包括PubMed文献和PMC全文。
BlueBERT: 由微软和亚马逊联合开发，基于BERT架构，预训练数据包括PubMed和MIMIC-III临床记录。
ClinicalBERT: 专门在临床文本（如MIMIC-III数据集）上进行预训练，适用于电子健康记录（EHR）和其他临床数据的自然语言处理任务。
2020年
PubMedBERT: 专门在PubMed摘要上进行预训练，适用于生物医学信息检索、问答和文献分类等任务。
2021年
BioMegatron: 基于NVIDIA的Megatron架构，经过大规模生物医学文献的预训练，支持高效并行计算，适用于处理大规模生物医学数据集。

This repository contains the live version of the Medical Abbreviation and Acronym Meta-Inventory, a large-scale American English database of medical abbreviations and their possible senses.
https://github.com/lisavirginia/clinical-abbreviations


Configuration option - show diagnostics for open files only

Currently it's for all files.
I think default should be open files only.
In any case, this should be configurable.

This is now the default behavior, no configuration available.


kv: Use Bytes as value to avoid copy between read/write

Now, we are store Vec<u8> in all kv adapters. Maybe it's better to store a Bytes instead?
For example, read data from HashMap<String, Bytes> will not need to clone the entire bytes anymore.

@PsiACE, do you wanna to take this issue a look?


[Bug]: Error in Laravel Extension (WSL2 Windows)

Extension Version
0.1.11
PHP Binary
Other
Operating System
Linux
What happened?
The extension fails to start on my machine. I use WSL2 (Ubuntu) within my Windows 11 machine.

Here are some errors logged:
[error] /bin/sh: 1: /home/hajx/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads/php-parser-0.1.6: Exec format error
[error] Error:
 Translation namespaces
 [error] /usr/bin/php -r "
define('LARAVEL_START', microtime(true));

Mimimal Code Sample
No response

I am getting this too on Linux Mint
[error] /bin/sh: 1: /home/****/.config/Code/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads/php-parser-0.1.6: Exec format error

I'm getting too in fedora
[error] /bin/sh: line 1: /home/****/.config/Code/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads/php-parser-0.1.6: cannot execute binary file: Exec format error

Well, looks like the internally used php-parser is arm64 only :(
❯ file php-parser-0.1.6
php-parser-0.1.6: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>

It would be a good idea to mention “macos arm only” if the extension is only designed for this platform. This would be a first for VSCode extensions. An oversight perhaps?

I created a workflow to build the binaries for macos (arm64 and x86) as well as linux x86.
https://github.com/SamuelNitsche/vs-code-php-parser-cli/blob/main/.github/workflows/build-release-binaries.yml
You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

I created a workflow to build the binaries for macos (arm64 and x86) as well as linux x86. https://github.com/SamuelNitsche/vs-code-php-parser-cli/blob/main/.github/workflows/build-release-binaries.yml
You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

Thanks! 🔥 That worked for me.
❯ ./php-parser-0.1.6

  VS Code PHP Parser  0.1.6

  USAGE:  <command> [options] [arguments]

  autocomplete Parse the given PHP code and return the autocomplete results
  detect       Detect things we care about in the current code

I created a workflow to build the binaries for macos (arm64 and x86) as well as linux x86. https://github.com/SamuelNitsche/vs-code-php-parser-cli/blob/main/.github/workflows/build-release-binaries.yml
You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

I'm getting a permission denied issue in vscode in plugin. Anyway quick fix to solve that? I'm using mac x86_64

I created a workflow to build the binaries for macos (arm64 and x86) as well as linux x86. https://github.com/SamuelNitsche/vs-code-php-parser-cli/blob/main/.github/workflows/build-release-binaries.yml
You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

I'm getting a permission denied issue in vscode in plugin. Anyway quick fix to solve that? I'm using mac x86_64

chmod +x

I'm using zorin os on Intel i7 my solution was download the file https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9
mv php-parser-0.1.9-x86_64-linux php-parser-0.1.6
chmod +x php-parser-0.1.6

After installing the php-parser built for Linux it stopped the error, but no autocomplete works. It always stuck Loading... indefinitely.

I'm using zorin os on Intel i7 my solution was download the file for linux from https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9
mv php-parser-0.1.9-x86_64-linux php-parser-0.1.6
chmod +x php-parser-0.1.6

One liner, change directory to where it is crashing: then do
wget "https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/download/0.1.9/php-parser-0.1.9-$(uname -m)-$(uname -s)" -O php-parser-0.1.6 && chmod +x php-parser-0.1.6

Note: temporary fix

i am using ubuntu i am also seeing the same error for php-parser-0.1.6.
this solution works though:

You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

but i won't go that way for now

I created a workflow to build the binaries for macos (arm64 and x86) as well as linux x86. https://github.com/SamuelNitsche/vs-code-php-parser-cli/blob/main/.github/workflows/build-release-binaries.yml
You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

worked for me too thanks.

I created a workflow to build the binaries for macos (arm64 and x86) as well as linux x86. https://github.com/SamuelNitsche/vs-code-php-parser-cli/blob/main/.github/workflows/build-release-binaries.yml
You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

Thank you sir!

i am using ubuntu i am also seeing the same error for php-parser-0.1.6. this solution works though:

You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

but i won't go that way for now

I'm running VS Code on Windows and using the Remote Development extension to connect to an Ubuntu 22.04 VM. Manually downloading php-parser stopped the php-parser errors but unfortunately the extension still crashes for me.

I got this error on Arch Linux, the problem is i mainly use Docker to run PHP app and there's no PHP binary executable on my local machine. So the extensions just throw the exception:
2024-12-17 10:26:14.403 [info] Activating Laravel Extension...
2024-12-17 10:26:14.403 [info] Started
2024-12-17 10:26:14.471 [error] /usr/bin/env: ‘php’: No such file or directory

Any solution for this?

I am running Vs Code on Mac and I am getting this error as well:

$app->register(new VsCodeLaravel($app));
$kernel = $app->make(Illuminate\Contracts\Console\Kernel::class);
$kernel->bootstrap();

echo '__VSCODE_LARAVEL_START_OUTPUT__';
 function vsCodeTranslationValue($key, $value, $file, $lines): array {     $lineNumber = 1;     $keys = explode('.', $key);     $index = 0;     $currentKey = array_shift($keys);      while ($currentKey !== null && $index < count($lines)) {         $line = $lines[$index][1];          if (strpos($line, '"' . $currentKey . '"', 0) !== false || strpos($line, "'" . $currentKey . "'", 0) !== false) {             $lineNumber = $lines[$index][0];             $currentKey = array_shift($keys);         }          $index++;     }      return [         'value' => $value,         'path' => realpath($file),         'line' => $lineNumber,         'params' => preg_match_all('/\:([A-Za-z0-9_]+)/', $value, $matches) ? $matches[1] : [],     ]; }  function vscodeCollectTranslations(string $path, string $namespace = null) {     return collect(glob("{$path}/**/*.{php,json}", GLOB_BRACE))->map(function ($file) use ($path, $namespace) {         $key = pathinfo($file, PATHINFO_FILENAME);          if ($namespace) {             $key = "{$namespace}::{$key}";         }          $lang = collect(explode('/', str_replace($path, '', $file)))->filter()->first();          $fileLines = explode(PHP_EOL, file_get_contents($file));         $lines = [];         $inComment = false;          foreach ($fileLines as $index => $line) {             $trimmed = trim($line);              if (substr($trimmed, 0, 2) === '/*') {                 $inComment = true;                 continue;             }              if ($inComment) {                 if (substr($trimmed, -2) !== '*/') {                     continue;                 }                  $inComment = false;             }              if (substr($trimmed, 0, 2) === '//') {                 continue;             }              $lines[] = [$index + 1, $trimmed];         }          return [             'key' => $key,             'lang' => $lang,             'values' => collect(\Illuminate\Support\Arr::dot(__($key, [], $lang)))->map(function ($value, $key) use ($file, $lines) {                 if (is_array($value)) {                     return null;                 }                  return vsCodeTranslationValue($key, $value, $file, $lines);             })->filter(),         ];     }); }  $loader = app("translator")->getLoader(); $namespaces = $loader->namespaces();  $reflection = new ReflectionClass($loader); $property = $reflection->hasProperty("paths")   ? $reflection->getProperty("paths")   : $reflection->getProperty("path"); $property->setAccessible(true);  $paths = \Illuminate\Support\Arr::wrap($property->getValue($loader));  $default = collect($paths)->flatMap(function ($path) {   return vscodeCollectTranslations($path); });  $namespaced = collect($namespaces)->flatMap(function ($path, $namespace) {     return vscodeCollectTranslations($path, $namespace); });  $final = [];  foreach ($default->merge($namespaced) as $value) {     foreach ($value['values'] as $key => $v) {         $dotKey = "{$value['key']}.{$key}";          if (!isset($final[$dotKey])) {             $final[$dotKey] = [];         }          $final[$dotKey][$value['lang']] = $v;          if ($value['lang'] === \Illuminate\Support\Facades\App::currentLocale()) {             $final[$dotKey]['default'] = $v;         }     } }  echo json_encode($final); ;
echo '__VSCODE_LARAVEL_END_OUTPUT__';

exit(0);

I got this error on Arch Linux, the problem is i mainly use Docker to run PHP app and there's no PHP binary executable on my local machine. So the extensions just throw the exception:
2024-12-17 10:26:14.403 [info] Activating Laravel Extension...
2024-12-17 10:26:14.403 [info] Started
2024-12-17 10:26:14.471 [error] /usr/bin/env: ‘php’: No such file or directory

Any solution for this?

I recommend you to have the php cli available in the host as many tools you may use rely on it. You can install it using asdf using the command: PHP_WITHOUT_PEAR=yes PHP_CONFIGURE_OPTIONS="--with-password-argon2 --with-openssl --with-readline" asdf install php 8.4.1

I too get this error

I am facing the same issue. I’m using Ubuntu.

I got this error on ubuntu linux

The same error "/bin/sh: 1: /home/user/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads/php-parser-0.1.6: Exec format error" on Windows WSL, VS Code Version: 1.96.0

I created a workflow to build the binaries for macos (arm64 and x86) as well as linux x86. https://github.com/SamuelNitsche/vs-code-php-parser-cli/blob/main/.github/workflows/build-release-binaries.yml
You can download the files from the release (https://github.com/SamuelNitsche/vs-code-php-parser-cli/releases/tag/0.1.9) and replace the file located at ~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads

Thanks. Working for me (VSCodium 1.96.0, on Debian 12)

same

I am also receiving this error, WSL2 on Windows.
Host: Windows 10 Pro - 22H2 - build 19045.5247
WSL2: Ubuntu 20.04.6 LTS
~/.vscode-server/data/User/globalStorage/laravel.vscode-laravel/file-downloader-downloads/php-parser-0.1.6

Still working on Windows, but there are new binary builds in 0.1.12, just released. Please let me know if it helps Mac/Linux users.

Working here now on Linux Mint!!!! Thank you for the awesome plugin and the quick fix!

Still working on Windows, but there are new binary builds in 0.1.12, just released. Please let me know if it helps Mac/Linux users.

looks like it work, but now i have this error:
2024-12-17 14:05:32.916 [error] /bin/sh: -c: line 2829: unexpected EOF while looking for matching ``'
/bin/sh: -c: line 4505: syntax error: unexpected end of file
MacOS

v.0.1.12 works on mac x86_64!

working on fedora

This is the new error I get after installing the new update.

Still working on Windows, but there are new binary builds in 0.1.12, just released. Please let me know if it helps Mac/Linux users.

The error message is changed to be:
2024-12-18 10:41:22.420 [info] Activating Laravel Extension... 2024-12-18 10:41:22.420 [info] Started 2024-12-18 10:41:28.680 [error]  2024-12-18 10:41:28.680 [error]  2024-12-18 10:41:28.698 [error]  2024-12-18 10:41:28.699 [error]  2024-12-18 10:41:28.699 [error]  2024-12-18 10:41:28.699 [error]
I use Linux distribution Lubuntu 24.04

working now as well. cool

I think mine is solved now, but now I have another error 😕

I think mine is solved now, but now I have another error 😕 something like Translation namespaces

After having some brute force testing remove some files, I think I found the issue.
I think it is because the lang files contain too long characters and reach over limit?

Just re-tried the extension with the new 0.1.13 update and now the problem is fixed! 🤩

Thank you @joetannenbaum! 🥰🙏


ownCloud Eight features page

@karlitschek @tomneedham and whoever else is interested... First draft of owncloud.org/eight ;-)
Rather than owncloud.org/seven I took the design of owncloud.org/features as @jancborchardt has been complaining about the look of that page.
Feedback on both text as well as design welcome. I'm waiting for feedback on the announcement draft, from which much of this text is taken, so that will bring some changes still. Just wanted to get this first draft out there.
Screenshot below (on 1080x1920 thus 90⁰ rotated screen):

you are fast! :+1:

Look is still largely the same. If @MorrisJobke and @jancborchardt are OK with it it looking this way, I think it can go to staging already...

Can we also added some bug fixing topics? Maybe something like "more robust external storage for webdav and SMB" or maybe @pvince84 knows some nice improvements there, that were problems in 7 but are fixed in 8, because otherwise it will again look like a complete feature-only release. Bug fixes and robustness are also essential features that can be promoted as well (if they are actually true ;))

@PVince81 just got younger! ;-)

@PVince81 ;) (it didn't autocomplete anymore vincent :()

:cry:

A few that come to mind:

mounting ownCloud into ownCloud as external storage now work properly with encryption
improved group share handling (when sharing a file multiple times with the same user over group+user) not 100% sure CC @schiesbn (might have been backported stuff)
encryption keys are now stored in folders which makes it easier/faster to move (more efficient code)
small performance improvement by caching encryption keys that are reused multiple times within a single PHP request

Hmm... I quickly browsed through the list of closed tickets and commits but didn't find that many that are interesting to expose. Also many were backported to OC 7.

@PVince81 Thanks. I just had you in mind, because you had done a lot "under the hood". ;) I guess this is enough for a smal teaser text and a new category of improvements, @jospoortvliet ;)

So much has happened, hard to remember.
One idea would be to have a "Changelog" wiki page where people can add whichever bigger fixes were closed, and do this as soon as the fix was closed, not several months later.
Not sure how @karlitschek usually proceeded to create changelogs so far ?

@PVince81 You're right. Maybe we can add this to the workflow. Once anything relevant is merged the author has to write it to the changelog wiki page.

Well, we could do stuff automated if people could agree on one standard for commit messages :) https://pypi.python.org/pypi/gitchangelog/2.2.0

@LukasReschke That doesn't work, because otherwise the commit head is full of qualifiers which component is effected :( And there are always people that didn't add proper commit messages :cry:

Rather than owncloud.org/seven I took the design of owncloud.org/features as @jancborchardt has been complaining about the look of that page.

@jospoortvliet thank you very much! Looks great! :) Only one thing: Can we please call the versions by numbers, that is owncloud.org/8 instead of writing them out as »Eight«? That just looks strange.

Then we need to call it 8.0, because 8.1 is no subset of 8 but a new major version.

FYI - am working on incorporating the comments ;-)

@PVince81 We don´t do changelogs for major version but only minor versions.

@MorrisJobke @jancborchardt ok, owncloud.org/8.0 it will be with the next commit.
@PVince81 look at the last item in the admin section - improvements and polish all over is what is claimed. I know there's quite some good performance and bugfix work (isn't there always?) and I'd love to talk it up as a feature but as mentioned, most is back ported to stable 7...
I have used @MTRichards text for it all, he wrote it more value-based and less concrete. We can debate if it should be more concrete but that's a conversation that should include him :D

actually, it has to be 8-0, not 8.0 due to a limitation in Wordpress (from what I can tell)...

Please don’t make it 8.0 / 8-0 :D that’s just technical details. owncloud.org/8 is much simpler, and that page will simply evolve over time if need be.

Please don’t make it 8.0 / 8-0 :D that’s just technical details. owncloud.org/8 is much simpler, and that page will simply evolve over time if need be.

But then people expect that 8.1 is just a minor release, but it is a major one :( Let's see

Yeah … so then we touch the problem with how we name versions. ;) I would rather have the URL stay owncloud.org/8, and then adjust the content of the page for 8.1, 8.2, 8.3
Don’t have a particular preference on this though. I just think the URL owncloud.org/8.0 or 8-0 is really ugly and techy.

Let's make it 8...

so that doesn't work in wordpress. @jancborchardt @LukasReschke @MorrisJobke it is either 8-0 or eight, I can't do 8.0 or 8 0 or 8. Choose... For now, i go with eight as that fits with what we did before.

I'd like to move this to staging to be able to get some actual review on the text...

Will make a new PR for further changes.

Yeah, works for me. Why does »8« not work with wordpress? Minimum character limit or what?

@jancborchardt yeah, something like that I suppose. I tried it locally and on our server and whatever I tried, a number (no matter how many digits) would always get a -0 (or -1 or -2) attached. Guess it's a wordpress thingy...

You can now see the state on http://staging.owncloud.org/eight/ - @jancborchardt @MorrisJobke any further feedback is welcome :D

I will propose some updated pictures ;)

@jospoortvliet I just shared to an external cloud and then right clicked the text and "inspect element" then you can change the text and don't have to put that ugly distortion filter onto it ;) (I have no nice data to be presented in the picture)

I think that is better than this:

:+1: * :100:

Also the User_LDAP has proper names ;)

Let me create a screenshot for that

@jospoortvliet Just use that one:

@jospoortvliet I believe >8< for the page url should be fine. Wordpress will auto change it, if you ahve another page with this name in your instance so this might be the issue.

@MorrisJobke We have: https://github.com/owncloud/screenshots/tree/stable8.0

https://cloud.morrisjobke.de/public.php?service=files&t=a9502fd4b25b86e36af3e3e71cf33094

I will create a PR to include these images

@jospoortvliet takes care of this :dancers:

All in the other PR: #411


Router is not "accepting" Browser/Hash history?

I'am trying to set up react router using v1.0.0-beta2. I've installed it via npm by using npm install react-router@beta and built by running ./scripts/build.sh. My code looks like this:
React = require 'react'
{Router, Route} = require 'react-router'
BrowserHistory = require 'react-router/lib/BrowserHistory'
routes = require './../shared/routes
router =
  <Router history={BrowserHistory} children={routes} />
React.render(router, document.getElementById('app-container'))

and I get following error:
Warning: Failed propType: Invalid prop `history` supplied to `Router`, expected instance of `History`.
browser.js:42 Uncaught Error: Invariant Violation: A <Router> needs a valid Location

So I thought that maybe I should pass instance of BrowserHistory to the router:
React = require 'react'
{Router, Route} = require 'react-router'
BrowserHistory = require 'react-router/lib/BrowserHistory'
routes = require './../shared/routes
router = # now passing instance!
  <Router history={new BrowserHistory} children={routes} />
React.render(router, document.getElementById('app-container'))

And I get same error again. Same is happening with HashHistory. What I am doing wrong?

The second code should work I think. Maybe you're somehow loading two instances of react-router? What does your build system look like?

You were right - my bad. I am using browserify to split code into app.js and vendor.js and forgot to add 'react-router/lib/BrowserHistory' as vendor lib
vendorLibs = [
  'react'
  'react-router'
  'react-router/lib/BrowserHistory'
]


Add support for Google's PaLM 2

I have access now. I managed to get an API key I can use with the text-bison-001 model via Google Vertex AI.
https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/text-bison
The API call looks like this:
url = "https://generativelanguage.googleapis.com/v1beta2/models/text-bison-001:generateText?key={}".format(
    api_key
)
response = requests.post(
    url,
    json={"prompt": {"text": prompt}},
    headers={"Content-Type": "application/json"},
)
output = response.json()["candidates"][0]["output"]

The thing I'm having trouble with here is what to call this.
I want a command similar to llm openai ... but for this family of models.
Naming options:

llm google
llm vertex (does anyone know what Vertex is?)
llm palm (but should this be palm2 and will there be non-palm models in here?)
llm bison (again, not sure people understand the terminology)

There are a ton of other models available in the Vertex "model garden": https://console.cloud.google.com/vertex-ai/model-garden - T5-FLAN, Stable Diffusion, BLIP, all sorts of things.
It's so very confusing in there! Many of them don't seem to have HTTP API endpoints - some appear to be available only via a notebook interface.

I'm tempted to go with llm google purely for consistency with llm openai - then maybe llm anthropic can follow?
 I've kind of broken this already though by having GPT-4 as a -4 flag on what I initially called the chatgpt command.

For PaLM 2 itself I think the models available to me are text-bison and code-bison and code-gecko.

I'm going to land a llm palm2 command for the moment, then go back to this issue and reconsider:

#17

This looks useful: https://developers.generativeai.google/api/rest/generativelanguage/models/list
curl https://generativelanguage.googleapis.com/v1beta2/models?key=$PALM_API_KEY

I currently get this:
{
  "models": [
    {
      "name": "models/chat-bison-001",
      "version": "001",
      "displayName": "Chat Bison",
      "description": "Chat-optimized generative language model.",
      "inputTokenLimit": 4096,
      "outputTokenLimit": 1024,
      "supportedGenerationMethods": [
        "generateMessage"
      ],
      "temperature": 0.25,
      "topP": 0.95,
      "topK": 40
    },
    {
      "name": "models/text-bison-001",
      "version": "001",
      "displayName": "Text Bison",
      "description": "Model targeted for text generation.",
      "inputTokenLimit": 8196,
      "outputTokenLimit": 1024,
      "supportedGenerationMethods": [
        "generateText"
      ],
      "temperature": 0.7,
      "topP": 0.95,
      "topK": 40
    },
    {
      "name": "models/embedding-gecko-001",
      "version": "001",
      "displayName": "Embedding Gecko",
      "description": "Obtain a distributed representation of a text.",
      "inputTokenLimit": 1024,
      "outputTokenLimit": 1,
      "supportedGenerationMethods": [
        "embedText"
      ]
    }
  ]
}

So no code-bison or code-gecko listed there.

https://developers.generativeai.google/api/rest/generativelanguage/models/countMessageTokens can count tokens:
curl https://generativelanguage.googleapis.com/v1beta2/models/chat-bison-001:countMessageTokens?key=$PALM_API_KEY \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
        "prompt": {
            "messages": [
                {"content":"How many tokens?"},
                {"content": "For this whole conversation?" }
            ]
        }
    }'

{
  "tokenCount": 23
}

I'm just going to ship text-bison-001 for the moment. I'm going to call the command palm2.

Where should it get the API token from? The way I do it for OpenAI tokens right now is bad and needs fixing:

#13

https://github.com/simonw/llm/blob/293f306a5e9855a0828f98293abbccf968810a6e/llm/cli.py#L162-L173

My code so far:

@cli.command()
@click.argument("prompt", required=False)
@click.option("-m", "--model", help="Model to use", default="text-bison-001")
@click.option("-n", "--no-log", is_flag=True, help="Don't log to database")
def palm2(prompt, model, no_log):
    "Execute a prompt against a PaLM 2 model"
    if prompt is None:
        # Read from stdin instead
        prompt = sys.stdin.read()
    api_key = get_vertex_api_key()
    url = "https://generativelanguage.googleapis.com/v1beta2/models/text-bison-latest:generateText?key={}".format(
        api_key
    )
    response = requests.post(
        url,
        json={"prompt": {"text": prompt}},
        headers={"Content-Type": "application/json"},
    )
    output = response.json()["candidates"][0]["output"]
    log(no_log, "vertex", None, prompt, output, model)
    click.echo(output)

    if prompt is None:
        # Read from stdin instead
        prompt = sys.stdin.read()

This poses a problem with the UX. This is the reason why llm happens to hang indefinitely.
I suggested a fix for this in PR#19.

    if prompt is None:
        # Read from stdin instead
        prompt = sys.stdin.read()

This poses a problem with the UX. This is the reason why llm happens to hang indefinitely when no argument is passed.

That's a deliberate design decision at the moment - it means you can run llm and then copy-and-paste text from your terminal.
There are other common unix commands that work like this - cat for example - so I'm not convinced it's a usability problem. Happy to hear further discussion around that though.

That's a deliberate design decision at the moment - it means you can run llm and then copy-and-paste text into your terminal.

I'm not sure to understand. When I run llm, no matter what I paste in my terminal, it just keeps waiting. Even if type "say hello" and press Enter.
Oh... my bad! Okay, I hit Ctrl + D and it responded. It's not very intuitive—though, I may be alone in that category.
As of now, I would find it better to print the helper message by default, as I suggested in my pull request. However, if an instruction can be shown to the user via stderr, it might solve what I believe is an UX issue.

Now that I've renamed llm chatgpt to llm prompt I'm going to try adding this model to that command instead, so you would use it like so:
llm -m palm2 "Five surprising names for a wise owl"

I'll support -m text-bison as well.

In that new prototype branch:
% llm 'Two names for a beaver' -m palm2
Daggett and Dasher

Figuring out chat mode for Vertex/PaLM2 is proving hard.
https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/1?project=cloud-vision-ocr-382418 talks about "PaLM 2 for Chat".
https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/generative_ai/chat.py seems to be the most relevant example code:
from vertexai.preview.language_models import ChatModel, InputOutputTextPair

def science_tutoring(temperature: float = 0.2) -> None:
    chat_model = ChatModel.from_pretrained("chat-bison@001")
    parameters = {
        "temperature": temperature,  # Temperature controls the degree of randomness in token selection.
        "max_output_tokens": 256,  # Token limit determines the maximum amount of text output.
        "top_p": 0.95,  # Tokens are selected from most probable to least until the sum of their probabilities equals the top_p value.
        "top_k": 40,  # A top_k of 1 means the selected token is the most probable among all tokens.
    }
    chat = chat_model.start_chat(
        context="My name is Miles. You are an astronomer, knowledgeable about the solar system.",
        examples=[
            InputOutputTextPair(
                input_text="How many moons does Mars have?",
                output_text="The planet Mars has two moons, Phobos and Deimos.",
            ),
        ],
    )
    response = chat.send_message(
        "How many planets are there in the solar system?", **parameters
    )
    print(f"Response from Model: {response.text}")
    return response

I think this is where vertexai comes from: https://github.com/googleapis/python-aiplatform

Buried deep in a class hierarchy, this looks like the code that actually constructs the JSON to call the API: https://github.com/googleapis/python-aiplatform/blob/c60773a7db8ce7a59d2cb5787dc90937776c0b8f/vertexai/language_models/_language_models.py#L697-L824
The API call then goes through this code:
        prediction_response = self._model._endpoint.predict(
            instances=[prediction_instance],
            parameters=prediction_parameters,
        )

I have not yet tracked down that self._model._endpoint.predict() method.

Some useful hints in https://github.com/google/generative-ai-docs/blob/main/site/en/tutorials/chat_quickstart.ipynb - including that PaLM 2 has a "context" concept which appears to be the same thing as an OpenAI system prompt:
reply = palm.chat(context="Speak like Shakespeare.", messages='Hello')
print(reply.last)

Hello there, my good fellow! How fares thee this day?

reply = palm.chat(context="Answer everything with a haiku, following the 5/7/5 rhyme pattern.", messages="How's it going?")
print(reply.last)

I am doing well
I am learning and growing
Every day is new
``

Based on that example notebook, I'm going to ditch the terminology "Vertex" and "PaLM 2" and just call it "PaLM". (They never released an API for PaLM 1).
I'm also going to move my code out of the experimental plugin and into a llm-palm package.

Got this out of the debugger:
(Pdb) pprint(response.to_dict())
{'candidate_count': None,
 'candidates': [{'author': '1',
                 'content': 'Here are three names for a dog:\n'
                            '\n'
                            '1. **Bailey** is a popular name for both male and '
                            'female dogs. It is of English origin and means '
                            '"bailiff" or "steward." Bailey is a friendly and '
                            'loyal dog that is always up for a good time.\n'
                            '2. **Luna** is a Latin name that means "moon." It '
                            'is a popular name for female dogs, but it can '
                            'also be used for male dogs. Luna is a beautiful '
                            'and intelligent dog that is always curious about '
                            'the world around her.\n'
                            '3. **Max** is a German name that means '
                            '"greatest." It is a popular name for male dogs, '
                            'but it can also be used for female dogs. Max is a '
                            'strong and courageous dog that is always willing '
                            'to protect his family.\n'
                            '\n'
                            'These are just a few of the many great names that '
                            'you could choose for your new dog. When choosing '
                            "a name, it is important to consider your dog's "
                            'personality and appearance. You should also '
                            'choose a name that you will be happy saying for '
                            'many years to come.'}],
 'context': '',
 'examples': [],
 'messages': [{'author': '0', 'content': 'three names for a dog'},
              {'author': '1',
               'content': 'Here are three names for a dog:\n'
                          '\n'
                          '1. **Bailey** is a popular name for both male and '
                          'female dogs. It is of English origin and means '
                          '"bailiff" or "steward." Bailey is a friendly and '
                          'loyal dog that is always up for a good time.\n'
                          '2. **Luna** is a Latin name that means "moon." It '
                          'is a popular name for female dogs, but it can also '
                          'be used for male dogs. Luna is a beautiful and '
                          'intelligent dog that is always curious about the '
                          'world around her.\n'
                          '3. **Max** is a German name that means "greatest." '
                          'It is a popular name for male dogs, but it can also '
                          'be used for female dogs. Max is a strong and '
                          'courageous dog that is always willing to protect '
                          'his family.\n'
                          '\n'
                          'These are just a few of the many great names that '
                          'you could choose for your new dog. When choosing a '
                          "name, it is important to consider your dog's "
                          'personality and appearance. You should also choose '
                          'a name that you will be happy saying for many years '
                          'to come.'}],
 'model': 'models/chat-bison-001',
 'temperature': None,
 'top_k': None,
 'top_p': None}

That library doesn't have streaming support yet, issue here:

https://github.com/google/generative-ai-python/issues/43

The context (PaLM's version of the system prompt) isn't very strongly held to:
Here's GPT-4:
% llm -m 4 'three names for a pet pelican, succinctly' --system 'translate to french'
trois noms pour un pélican domestique, succinctement

PaLM messes that one up:
% llm -m palm 'three names for a pet pelican, succinctly' --system 'translate to french'
{'model': 'models/chat-bison-001', 'context': 'translate to french', 'examples': [], 'messages': [{'author': '0', 'content': 'three names for a pet pelican, succinctly'}, {'author': '1', 'content': 'Here are three names for a pet pelican:\n\n* Pete\n* Piper\n* Peanut'}], 'temperature': None, 'candidate_count': None, 'top_p': None, 'top_k': None, 'candidates': [{'author': '1', 'content': 'Here are three names for a pet pelican:\n\n* Pete\n* Piper\n* Peanut'}]}
Here are three names for a pet pelican:

* Pete
* Piper
* Peanut
% llm -m palm 'three names for a pet pelican, succinctly' --system 'you are a bot that translates everything to french'
{'model': 'models/chat-bison-001', 'context': 'you are a bot that translates everything to french', 'examples': [], 'messages': [{'author': '0', 'content': 'three names for a pet pelican, succinctly'}, {'author': '1', 'content': 'Here are three names for a pet pelican:\n\n* Pete\n* Piper\n* Peanut'}], 'temperature': None, 'candidate_count': None, 'top_p': None, 'top_k': None, 'candidates': [{'author': '1', 'content': 'Here are three names for a pet pelican:\n\n* Pete\n* Piper\n* Peanut'}]}
Here are three names for a pet pelican:

* Pete
* Piper
* Peanut

This example from the PaLM example notebook does work though:
% llm -m palm 'Hello' --system 'Speak like Shakespeare'
{'model': 'models/chat-bison-001', 'context': 'Speak like Shakespeare', 'examples': [], 'messages': [{'author': '0', 'content': 'Hello'}, {'author': '1', 'content': 'Hello there, my good fellow! How fares thee on this fine day?'}], 'temperature': None, 'candidate_count': None, 'top_p': None, 'top_k': None, 'candidates': [{'author': '1', 'content': 'Hello there, my good fellow! How fares thee on this fine day?'}]}
Hello there, my good fellow! How fares thee on this fine day?

This was surprising:
llm -m palm --system "Translate to french" "I like pelicans a lot"

{'model': 'models/chat-bison-001', 'context': 'Translate to french', 'examples': [], 'messages': [{'author': '0', 'content': 'I like pelicans a lot'}, None], 'temperature': None, 'candidate_count': None, 'top_p': None, 'top_k': None, 'candidates': []}
I got back a None where I expected a message - and response.last return None too.

Extracted this out to a separate plugin: https://github.com/simonw/llm-palm
Closing this issue - future work will happen there instead.


A new trace is created instead of a child span when using subscribeOn(...) operator.

bug-report-1657.zip
Spring Boot: 2.2.7.RELEASE
Spring Cloud: Hoxton.SR4 (spring-cloud-sleuth-core-2.2.2.RELEASE)
When using operator subscribeOn(boundedElastic) a new trace is created instead of spawning a child span.
Here is a short example:
@RestController
public class FooController {

	private final Bar bar;

	public FooController(Bar bar) {
		this.bar = bar;
	}

	@GetMapping("/foo")
	public String foo() {
		return Mono.fromCallable(bar::bar)
		.subscribeOn(Schedulers.boundedElastic())
		.block();
	}
}

@Component
public class Bar {

	Tracer tracer;

	public Bar(Tracer tracer) {
		this.tracer = tracer;
	}

	@NewSpan
	public String bar() {
		return tracer.currentSpan().context().toString();
	}
}

When running such an application on localhost:8080 then when invoking http://localhost:8080/foo the response will always be parentId/spanId where parentId==spanId which indicate new trace. But it is expected it returns different parentId from spanId as it would do if there was no subscribeOn(Schedulers.boundedElastic()) operator in the FooController.foo() method.
I identified the problem being at line 55 of  SleuthContextListener which sometimes creates new instance of SleuthContextListener even when it should return an existing instance. The new instance always reports the context hasn't been refreshed. In certain cases the method will not find the existing instance in the CACHE because it was put to the cache on line 75 as
if (context instanceof ConfigurableApplicationContext) {
  beanFactory = ((ConfigurableApplicationContext) context).getBeanFactory();
}

The method getBean(BeanFactory) should also look like:
static SleuthContextListener getBean(BeanFactory beanFactory) {
  if (beanFactory instanceof ConfigurableApplicationContext) {
    beanFactory = ((ConfigurableApplicationContext) beanFactory).getBeanFactory();
  }
  return CACHE.getOrDefault(beanFactory, new SleuthContextListener());
}

This incorrect behavior causes ContextUtil.isContextUnusable(BeanFactory) to report the context is unusable even though the context was refreshed and is not closed which changes behavior in many places. For example the TraceableScheduledExecutorService.schedule method at line 49 where it instead of scheduling the TraceRunnable schedules the raw Runnable which causes a new trace is created instead of creating a child span.
bug-report-1657.zip

Hi! Thanks for the pointers. Are you wiling to create a failing test and a fix against the 2.2.x branch?


Make it possible to choose which layers to log or not log in modin logger

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. What kind of performance improvements would you like to see with this new API?
By default the Modin logger logs information about all layers that inherit ClassLogger this logs a lot of content, It would be good if  it would be  possible to choose which layers to log with modin logger.
By default modin logger could log all information to the file and if required user through environment variables can suggest which layers  Log if logs of only specific layers are required, For example this could be helpful if the user needs to log only higer level layers and leave out lower layers from being logged.

@anmyachev, @dchigarev, thoughts?

For example this could be helpful if the user needs to log only higer level layers and leave out lower layers from being logged.

enable_api_only could be used for that, right?
https://github.com/modin-project/modin/blob/c75343687799e3b69fc04a99b8a9d11eab1fd984/modin/config/envvars.py#L476

With enable_api_only currently all the classes that inherit classLogger would be logged, The only difference between enable_api_only and enable is in enable additionally memory.log also would be written.
As you said I think we could use this enable_api_only Variable to choose which layers to log.

With enable_api_only currently all the classes that inherit classLogger would be logged, The only difference between enable_api_only and enable is in enable additionally memory.log also would be written.

Also this:
https://github.com/modin-project/modin/blob/c75343687799e3b69fc04a99b8a9d11eab1fd984/modin/logging/config.py#L172
Maybe it would be enough to simply move the lower-level functionality logging to the debug level (instead of info)?

As you suggesting to use LogMode to choose which layers to log?

As an option. BTW how many different levels are there now?

Currently layer names have not been named, so every layer is just given the default name PANDAS-API, We could add the following layers.
Dataset
QueryCompiler
Dataframe
PandasDataframePartitionManager
Axis Partition
Block partition
Maybe a common layer for classes such as Dispatcher,parser, acessor etc

It seems to me enable_api_only and enable options look confusing. Maybe we should just leave enable option, using which memory.log would also be generated?

Maybe it would be enough to simply move the lower-level functionality logging to the debug level (instead of info)?

It would make sense since I think only developers would look at the lower-level functionality. Making it possible to choose which layers to log would only complicate the things for users. Thoughts?

It seems to me enable_api_only and enable options look confusing. Maybe we should just leave enable option, using which memory.log would also be generated?

I think this makes better sense as the memory.log would not be a large file as logging is just done every 5 seconds by default.

Maybe it would be enough to simply move the lower-level functionality logging to the debug level (instead of info)?

Which layers should be considered lower level?

Making it possible to choose which layers to log would only complicate the things for users. Thoughts?

Then can I close this issue if it would not be beneficial?

It seems to me enable_api_only and enable options look confusing. Maybe we should just leave enable option, using which memory.log would also be generated?

I think this makes better sense as the memory.log would not be a large file as logging is just done every 5 seconds by default.

Let's resolve that in a separate issue. Please create one.

Maybe it would be enough to simply move the lower-level functionality logging to the debug level (instead of info)?

Which layers should be considered lower level?

I think everything besides the API layer.

Making it possible to choose which layers to log would only complicate the things for users. Thoughts?

Then can I close this issue if it would not be beneficial?

Yes, let's close it.


Add items with Drag and Drop adapter

Hey, awesome library!
Is there a built-in way to add items to the drag and drop adapter?
Simply doing
list.add(i, string);
mWrappedAdapter.notifyItemInserted(i);

doesn't do anything.

Hi. Please change mWrappedAdapter to mAdapter. Wrapper adapters can automatically relay notify* events of the underlying adapter(s).

Hi thanks for the reply.
That won't work either, may the problem be with the data provider? The one I'm using is the one in the drag and drop basic example.

Maybe there is a trivial implementation bug in your adapter or data provider.

The one I'm using is the one in the drag and drop basic example.

Okay, here is a code snippet to add item insertion functionality to the basic D&D demo. Could you compare with your adapter/data provider implementation? Thanks.
class DraggableExampleItemAdapter {
    @Override
    public void onBindViewHolder(MyViewHolder holder, int position) {
        holder.mContainer.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                RecyclerView rv = RecyclerViewAdapterUtils.getParentRecyclerView(v);
                RecyclerView.ViewHolder vh = rv.findContainingViewHolder(v);
                int position = vh.getAdapterPosition();

                mProvider.addNewItem(position);
                notifyItemInserted(position);
            }
        });

       ...
    }
   ...
}

public abstract class AbstractDataProvider {
    public void addNewItem(int position) {
    }
    ...
}

public class ExampleDataProvider {
    @Override
    public void addNewItem(int position) {
        // XXX: Need to improve ID generation code, because using mData.size() won't
        // work properly if once `removeItem()` is called.
        mData.add(position, new ConcreteData(mData.size(), 0, "THIS IS ADDED ITEM", 0));
    }
}

You saved me days with this library, I'd love to see a donations button in the README soon.
:)


CONTRIBUTING: add guidelines for commit msgs

To help contributors, I added guidelines and examples for commit messages. This came from the discussion in #844.
I welcome any comments or suggestions.
ping @ebfe @Heather @Arcterus

@jbcrail looks good for me


(Bug) When creating a new account, phone and email are not added to the profile

Need to make sure that:
a) storage initialized before signup
b) and setProfile calls have performed without any issues before finish registration
Description

add retry userStorage.init() and setProfile(),
show error dialog if they're failing again,
if userStorage initilization failed twice, display reload app button

About #2121
How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
Checklist:

[x] PR title matches follow: (Feature|Bug|Chore) Task Name
[x] My code follows the style guidelines of this project
[x] I have followed all the instructions described in the initial task (check Definitions of Done)
[x] I have performed a self-review of my own code
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I have added reference to a related issue in the repository
[x] I have added a detailed description of the changes proposed in the pull request. I am as descriptive as possible, assisting reviewers as much as possible.
[ ] I have added screenshots related to my pull request (for frontend tasks)
[ ] I have pasted a gif showing the feature.
[x] @mentions of the person or team responsible for reviewing proposed changes

in indexProfileField  need to remove this.trust usage
this.trust are indexes managed by our own gun user, via goodserver
const indexNode = this.gun.get(this.trust[`by${field}`] || `users/by${field}`).get(cleanValue)
to
const indexNode = this.gun.get(`users/by${field}`).get(cleanValue)


Unable to download node

$git push deis master
-----> Installing binaries
engines.node (package.json):  4.2.x
engines.npm (package.json):   unspecified (use default)
   Resolving node version 4.2.x via semver.io...
   Downloading and installing node ...
   Unable to download node ; does it exist?

-----> Build failed
   We're sorry this build is failing! You can troubleshoot common issues here:
   https://devcenter.heroku.com/articles/troubleshooting-node-deploys

   If you're stuck, please submit a ticket so we can help:
   https://help.heroku.com/

   Love,
   Heroku

@reutopiaer What version of deis are you using? Did you get this error when deploying the example nodejs app?

$deis -v
2.2.0-32177aa�
$git push deis master�
Counting objects: 195, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (88/88), done.
Writing objects: 100% (195/195), 35.34 KiB | 0 bytes/s, done.
Total 195 (delta 104), reused 195 (delta 104)
Starting build... but first, coffee!                                           ����������������������������������������������������������������������������������������������������������������������������������
...
-----> Node.js app detected
-----> Creating runtime environment
   NPM_CONFIG_LOGLEVEL=error
   NPM_CONFIG_PRODUCTION=true
   NODE_ENV=production
   NODE_MODULES_CACHE=true

-----> Installing binaries
engines.node (package.json):  4.2.x
engines.npm (package.json):   unspecified (use default)
   Resolving node version 4.2.x via semver.io...
   Downloading and installing node ...
   Unable to download node ; does it exist?

-----> Build failed
   We're sorry this build is failing! You can troubleshoot common issues he

e:
https://devcenter.heroku.com/articles/troubleshooting-node-deploys
   If you're stuck, please submit a ticket so we can help:
   https://help.heroku.com/

   Love,
   Heroku�

9 times out of 10 this is a connection issue/DNS issue. When I hit this problem, the DNS server provided by my ISP redirected the requests incorrectly.
What provider are you using to install Kubernetes, and what version of Kubernetes are you running? Is this on Vagrant or on a public cloud provider like AWS?

local area network   kubernetes/cluster/vagrant   https://deis.com/docs/workflow/quickstart/provider/vagrant/install-vagrant/

opps，I made a big mistake，when I stop dnsmsaq the problem is solved. Sorry for everyone!


emerald work area

import from https://github.com/oasisprotocol/oasis-nexus/pull/1

big, chunky todos

improve testing to be as easy as it was for early days oasis-nexus
file away items from analyzer/uncategorized into appropriate locations
file away visitor handler logic into analyzer/modules
unite block_extra and transaction_extra with existing tables
spread small things like CloseOrLog around to rest of repo

Nice!

big, chunky todos

Do you intend to address these in a follow-up PR or in this one? I'm in favor of the former, but either way. Just so long as the scope of reviewing is clear.

yeah I'll work on them in this pr
On Fri, Sep 30, 2022, 2:45 PM mitjat @.***> wrote:

Nice!
big, chunky todos
Do you intend to address these in a follow-up PR or in this one? I'm in
favor of the former, but either way. Just so long as the scope of reviewing
is clear.
—
Reply to this email directly, view it on GitHub
https://github.com/oasisprotocol/oasis-indexer/pull/141#issuecomment-1264052496,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJDVILUVHID2ESSFYSC4SFDWA5NRBANCNFSM6AAAAAAQWD3IBM
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
Disclaimer: This e-mail and any attachments may contain confidential
information. If you are not the intended recipient, any disclosure,
copying, distribution or use of any information contained herein is
strictly prohibited. If you have received this transmission in error,
please immediately notify the sender and destroy the original transmission
and any attachments without reading or saving.

here's my plan for integrating nexus db table/indexes:
db unification

oasis_3.emerald_rounds

block_extra.num_transactions will go into extra_data
block_extra.gas_used will go into extra_data

this will somewhat duplicate oasis_3.emerald_gas_used

block_extra.size will go into extra_data
unordered index added on block_hash

oasis_3.emerald_transactions

new table

indexer currently has separate tables for different transaction-related events
I believe we had always desired to have a table of transactions as well, just it had not been implemented
based on transaction_extra from nexus, although in indexer style, there is no chain_alias column, and Emerald gets its own table

height numeric

height is the round, but indexer calls it height
nexus used bigint for round. is it more expensive to index numeric columns?

tx_index integer
primary key (height, tx_index)
tx_hash text

nexus used char(64) for hashes. is it more expensive to index text columns for lookup by equality?

eth_hash text null
is there desire for an extra_data json column?
is there desire for a bytea of the actual transaction?

I see consensus oasis_3.transaction has body bytea, was this really only the body?
note that to

unordered index on tx_hash
unordered index on eth_hash
no connection with existing emerald_transfers, ... etc event-specific tables :thinking:

oasis_3.emerald_transaction_signer

new table

justification: some interactions, notably deposit+withdraw results, need to be looked up by signer and nonce
based on transaction_signer from nexus

height numeric
tx_index integer
signer_index integer
primary key (height, tx_index, signer_index)
addr text
nonce integer
index on (addr, nonce)

oasis_3.emerald_related_transaction

new table

justification: for showing a list of related transactions for a given account
based on related_transaction from nexus
may alternatively be possible by querying many event tables and merging results?
this approach preserves transaction order though
this approach leaves out non-transaction events

account_address text
tx_height numeric
tx_index integer
index on (account_address, tx_height, tx_index)

oasis_1.address_preimage

new table

justification: determine ethereum address corresponding to oasis address, explorer UI will probably use only ethereum addresses where possible
justification: determine public key corresponding to oasis address, for registry lookups
namespace is oasis_1, referring to the earliest hardfork using compatible address derivation schemes
don't forsee hardforks ever to change the derivation scheme; shouldn't need to move to a new table unless it does
analysis of older hardforks can benefit from information obtained while analyzing newer hardforks
data collected is not isolated to Emerald runtime

address text primary key
context_identifier text
context_version integer
addr_data bytea

oh I read that wrong, you prefer to address these in separate PRs? that may be fine, as long as we have an understanding that we'll be editing the 0011 migration instead of doing further migrations on top. it's just too rough right now imo

Oh that's a good writeup on table migrations :ok_hand: . I copy-pasted it into https://docs.google.com/document/d/1moqmGcHosdnVUHfYT6M4mpQbWsWEnmHIOpFAfcosZdY/edit#heading=h.imth7ku5x6is (because github is not great for commenting on comments piecemeal), and added comments. Not 100% done yet but dinner calls; will wrap up probably Monday.

ok let's do this in separate PRs

table migrations

We did agree (verbally I believe) to squash current migrations.

reviewers, can we merge this?

let's do most things in #158 or later. that PR will be moving things around, so it would be more work to adjust things here and rebase #158

Error: cannot use batch (variable of type *storage.QueryBatch) as *pgx.Batch value in argument to emitRoundBatch (typecheck)
ugh what's going on in main