Warp run returns error "The specified key does not exist"

Environment info

NooBaa Version:

[root@hpo-app11 ~]# noobaa version
INFO[0000] CLI version: 5.10.1
INFO[0000] noobaa-image: noobaa/noobaa-core:5.10.1
INFO[0000] operator-image: noobaa/noobaa-operator:5.10.1

Platform:  4.10.16

Actual behavior

Error The specified key does not exist is returned while running warp and performing I/O

Expected behavior

Warp run and I/O should complete without any issues

Steps to reproduce

Start warp run simultaneously through all 3 IP's for long duration
Do failover of noobaa-core pod
Check that IP movement is done during failover and warp run does not get aborted.
Once failover is done, test failback
In failback all pods should come up and warp should not affected
In failback warp gets aborted and returns "The specified key does not exist" error. Also I/O fails with this error

More information - Screenshots / Logs / Other output
Output of warp run
[root@hpo-app11 ~]# warp mixed --host=<IP_ADDRESS> --access-key=jotSmTsCPFOCOtjoBhXG --secret-key=iBvPp8xQ1oR6SmPr/iWC6Td4xSm4OtjC/9atIEnP --obj.size=50m --d$
ration=82m --bucket=bucket-for-export2 --debug --insecure --tls
warp: <ERROR> The specified key does not exist.
warp: <ERROR> upload error: The specified key does not exist.
warp: <ERROR> upload error: The specified key does not exist.
warp: <ERROR> upload error: The specified key does not exist.
warp: <ERROR> upload error: The specified key does not exist.
warp: <ERROR> upload error: The specified key does not exist.
...
panic: ran out of objects
goroutine 2866 [running]:
github.com/minio/warp/pkg/bench.(*MixedDistribution).randomObj(0xc000167f80)
..
created by github.com/minio/warp/pkg/bench.(*Mixed).Start
        github.com/minio/warp/pkg/bench/mixed.go:235 +0x12f

Output of I/O
[root@hpo-app11 ~]# s3user2 cp /root/awscli-bundle.zip s3://bucket-for-export2
/usr/local/aws/lib/python3.6/site-packages/urllib3/connectionpool.py:1050: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
upload failed: ./awscli-bundle.zip to s3://bucket-for-export2/awscli-bundle.zip An error occurred (NoSuchKey) when calling the CreateMultipartUpload operation: The specified key does not exist.

Please find the attached logs for reference.
noobaa.zip

This issue is the duplicate of https://github.com/noobaa/noobaa-core/issues/6968.
Warp run and I/O fails on the node as file system is not mounted on the particular noobaa endpoint.


Feat - Included ok-to-test feature

Included https://github.com/imjohnbo/ok-to-test similar prow.
This is being used in Scorecard.

ping @dlorenc


MH-12937: Correctly place admin UI test helper

I forgot to move a file when retargeting #255
from r/3.x to r/5.x.
This fixes that.

In my head this is a feature, so it should go into develop, right?
But I can of course retarget it if you want, I have no hard opinion on that. Just say the word. :slightly_smiling_face:

I thought that you have applied a PR to r/3.x and merged this upwards to r/4.x and r/5.x but forgot about the folder structure between r/4.x and r/5.x are different.
How can this be a feature?

See my edit, I thought we were talking about another PR. ;)

done


[Error: Failed to ingest your data]

creating vector store...
error [Error: PineconeClient: Project name not set. Call init() first.]
g:\gpt4-pdf-chatbot-langchain\scripts\ingest-data.ts:51
throw new Error('Failed to ingest your data');
^
[Error: Failed to ingest your data]

check if the file path is correct or not.

I have a similar question
error [Error: PineconeClient: Error calling upsert: Error: PineconeClient: Error calling upsertRaw: FetchError: The request failed and the interceptors did not return an alternative response]
/Users/liuchenghui01/Development/JavaScript/gpt4-pdf-chatbot-langchain/scripts/ingest-data.ts:51
throw new Error('Failed to ingest your data');
^

I also encountered this problem, which occurred randomly,

The file path is correct.
    metadata: { source: 'MorseVsFrederick.pdf' }
  },
  Document {
    pageContent: 'its educational mission to include solidarity with our soldiers \n' +
      'and  their  families  and  thus  could  have  attempted  to  outlaw \n' +
      'the  wearing  of  black  armbands  on  the  ground  that  they  un\n' +
      'dermined  this  mission.    Alternatively,  a  school  could  have \n' +
      'defined  its  educational  mission  to  include  the  promotion  of \n' +
      'world  peace  and  could  have  sought  to  ban  the  wearing  of \n' +
      'buttons  expressing  support  for  the  troops  on  the  ground \n' +
      'that the buttons signified approval of war.    The “educational \n' +
      'mission”  argument  would  give  public  school  authorities  a \n' +
      'license   to   suppress   speech   on   political   and   social   issues \n' +
      'based  on  disagreement  with  the  viewpoint  expressed.    The \n' +
      'argument,  therefore,  strikes  at  the  very  heart  of  the  First \n' +
      'Amendment.',
    metadata: { source: 'MorseVsFrederick.pdf' }
  },
  ... 78 more items
]
creating vector store...
error [Error: PineconeClient: Project name not set. Call init() first.]
g:\gpt4-pdf-chatbot-langchain\scripts\ingest-data.ts:51
    throw new Error('Failed to ingest your data');
          ^
[Error: Failed to ingest your data]
云飞扬
@.***

------------------ 原始邮件 ------------------
发件人:                                                                                "mayooear/gpt4-pdf-chatbot-langchain"                                                                                @.>;
发送时间: 2023年3月24日(星期五) 晚上9:03
@.>;
<EMAIL_ADDRESS>主题: Re: [mayooear/gpt4-pdf-chatbot-langchain] [Error: Failed to ingest your data] (Issue #46)
check if the file path is correct or not.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

I have a similar question， already checked file path, Pineone config: index name, dimensions value
error [Error: PineconeClient: Error calling upsert: Error: PineconeClient: Error calling upsertRaw: FetchError: The request failed and the interceptors did not return an alternative response] /Users/liuchenghui01/Development/JavaScript/gpt4-pdf-chatbot-langchain/scripts/ingest-data.ts:51 throw new Error('Failed to ingest your data'); ^

is your file size above 2 mb ?

I didn't have access to gpt-4 so I resolved this error by reducing pdf pages ingested, chunk size and overlap.

I didn't have access to gpt-4 so I resolved this error by reducing pdf pages ingested, chunk size and overlap.

Thank you for your answer, that's probably the reason.

I didn't have access to gpt-4 so I resolved this error by reducing pdf pages ingested, chunk size and overlap.

how do you set chunk size and overlap?

I have a similar question， already checked file path, Pineone config: index name, dimensions value
error [Error: PineconeClient: Error calling upsert: Error: PineconeClient: Error calling upsertRaw: FetchError: The request failed and the interceptors did not return an alternative response] /Users/liuchenghui01/Development/JavaScript/gpt4-pdf-chatbot-langchain/scripts/ingest-data.ts:51 throw new Error('Failed to ingest your data'); ^

is your file size above 2 mb ?

mine is, why? should be less than 2 mb?


cve fix

Signed-off-by: Xiangjing Li<EMAIL_ADDRESS>https://github.com/stolostron/backlog/issues/18845

/test test-unit

/test test-unit


Modified it to work with blank entries

Modified it to work with blank entries, for example, files without an
extension.
Also fixed two minor spelling errors in comments.

Thanks for catching the typos. Do you have a use case I can test with before I merge the pull request?

I was trying to get a count of files by extension, including files that did not have an extension:
$ > $files = gci e:\Scripts -File -Recurse
$ > New-PSPivotTable $files -yProperty Directory -xLabel Extension -Count | ft -AutoSize
Directory                           [NONE] .MD .DLL .PS1XML .PSD1 .JSON .LOG .BAK .PS1

E:\Scripts                               0   0    0       0     0     0    0    0   71
E:\Scripts\Get-PositionalParameters      1   1    0       0     0     0    0    0    1
E:\Scripts\JumpLocation                  0   2    1       2     1     0    0    0    3
E:\Scripts\NameGenerator                 0   0    0       0     0     0    0    0    1
E:\Scripts\PoshServerRoot                0   0    0       0     0     1    2    0    7
E:\Scripts\PShr                          1   1    0       0     0     0    0    0    1
E:\Scripts\PSNotes                       1   1    0       0     0     1    0    0    1
E:\Scripts\Sandbox                       0   0    0       0     0     0    0    0    8
E:\Scripts\Settings                      0   0    0       0     0     2    0    2    0

Ok. I think that can work. Thanks for the contribution and my first ever pull request.


Yarn workspace support

Would be great to get Yarn workspace support, is there any ETA? Depfu is not OS is it? Otherwise I would create a PR.

Hey @Siilwyn 👋
Thanks for the input, it's on our list and we should get to it quite soon. I'll let you know!

Alright cheers! Eagerly waiting for this to land. :)

Hey @Siilwyn: we've shipped a first version of this and it's looking good so far. Would be great to get your input on if this works as you expect it.

:clap:  Looks good! Only I can't find how to delete the 'npm' context of the project since I switched to yarn it isn't needed anymore. But I guess that would be a new separate feature.
Will close this and report here if I find strange behaviour.

@Siilwyn We're actually working on that right now unrelated to this feature, but I've cleaned it up for you manually now.


No such file or directory K8s Docker for Windows

Using mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04 I am trying to deploy SQL Server to my local cluster. For some reason it immediately fails with a bug check error (0x00000006) and an error that a file is not found. It eludes me what file is missing. The relevant parts of my yaml are as follows below. How can I diagnose what file is missing, if any?
apiVersion: v1
kind: PersistentVolume
metadata:
 name: mssqldb
 labels:
   type: local
spec:
 storageClassName: mssql-storage
 capacity:
   storage: 100Gi
 accessModes:
   - ReadWriteOnce
 hostPath:
   path: /C/K8sMounts/Datastore
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mssql-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: Immediate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mssqldb-claim
spec:
  storageClassName: mssql-storage
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
---
...
      containers:
        - name: {{ .Chart.Name }}
          securityContext:
            {{- toYaml .Values.securityContext | nindent 12 }}
          image: "{{ .Values.image.repository }}:{{ .Chart.AppVersion }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: tcp
              containerPort: 1433
              protocol: TCP
          env:
          - name: ACCEPT_EULA
            value: "Y"
          - name: SA_PASSWORD
            value: "Sq1Sup3r"
          - name: MSSQL_PID
            value: "Developer"
          volumeMounts:
          - name: mssqldb
            mountPath: /var/opt/mssql

From the logs:
The following diagnostic information is available:

         Reason: 0x00000006
        Message: Kernel bug check
        Address: 0x6ae47d60
     Parameters: 0x10861f590
    Stack Trace:
                 000000006af3543e
                 000000006ae47dbb
                 000000006ae3447e
                 000000006ae43025
                 000000006ae431f6
                 000000006af3370c
                 000000006af3227f
                 000000006af75671
        Process: 8 - sqlservr
         Thread: 12 (application thread 0x4)
    Instance Id: 3cc01a45-04d9-4333-880c-dbe5e86292a6
       Crash Id: 162e14a8-fbca-4850-86c8-c2baaf02fc66
    Build stamp: 52ec5c991c015cbdd504002245460be8a9a9b6c41343aaab03cf768750a6c2df
   Distribution: Ubuntu 16.04.6 LTS
     Processors: 5
   Total Memory:<PHONE_NUMBER>6 bytes
      Timestamp: Mon Nov 25 13:04:34 2019
     Last errno: 2
Last errno text: No such file or directory

Reading further, this https://github.com/microsoft/mssql-docker/issues/441#issuecomment-533057362 provided the answer for me. Please update the documentation here  https://docs.microsoft.com/en-us/sql/linux/tutorial-sql-server-containers-kubernetes?view=sql-server-ver15 with the right mount path.


[ISSUE #4778] Design of optimize log transfer protocol

[ISSUE #4778] Design of optimize log transfer protocol

Cool, thanks for your work. @lizhimins
Can you provide a performance comparison and stability test of the old and new versions?
If we switch to the netty version, I'm more curious about how much performance can be improved and how stable it is


Refactor system into separate files

Currently the system file includes memory, load average, and cpu functions, but I think it would be good to separate those out. gopsutil separates them and it makes it easier to organize things. However, this would be a breaking change.
edit: gopsutil has a host module that includes things like uptime, which is also in the system file. So maybe we could only include stuff like that and maybe rename it to 'host'.

It should be possible to make these changes without breaking the API by moving functions into separate files while still exporting them in the files they were originally in.
https://doc.rust-lang.org/1.4.0/book/crates-and-modules.html#re-exporting-with-pub-use


Massive slowdown between versions when calling backend.target.qubit_properties

Describe the bug
In 0.26.0 calling backend.target.qubit_properties takes much longer than previous versions.
Steps to reproduce
from qiskit_ibm_runtime import QiskitRuntimeService
from time import time

sevice_name = "<input-service-name>"
backend_name = "ibm_brisbane"
runtime = QiskitRuntimeService(name=sevice_name)
backend = runtime.get_backend(backend_name)

start = time()
[backend.target.qubit_properties[i] for i in range(127)]
print("Time taken for backend: ", time() - start)

Running the above snippet with older versions (tested with 0.21.2 and 24.1) is significantly faster than with 0.26.0. For older versions it takes about 1.5-2 seconds and with 0.26.0 it takes roughly 30-35 seconds.
Expected behavior
The time to be roughly the same between versions.
Additional Information
from qiskit_ibm_runtime import QiskitRuntimeService
from time import time

sevice_name = "<input-service-name>"
backend_name = "ibm_brisbane"
runtime = QiskitRuntimeService(name=sevice_name)
backend = runtime.get_backend(backend_name)

start = time()
target = getattr(backend, "target", None)
[target.qubit_properties[i] for i in range(127)]
print("Time taken for target: ", time() - start)

Running the above snippet only takes about 2 seconds on all tested versions (including 0.26.0).

qiskit-ibm-runtime version:
0.26.0, 0.21.2, 0.24.1

Python version:

3.9

Operating system:

OSX Sonoma 14.5

Hi @chookity-pokk, the backend target is no longer cached but you can just store the target in a variable.
qubit_properties = backend.target.qubit_properties
[qubit_properties[i] for i in range(127)]

This should only take a couple seconds

@kt474 Thank you for the clarification! Since this is expected behavior I'll close the issue.


Browse... button is missing from User Dictionary field on "Flat to XML" tab

There should be a "Browse..." button for selecting a User Dictionary file when converting a flat to XML file

I am pretty sure it's a Window sizing issue; you should be able to see them if you resize your Window. This should already be fixed in version 0.7 (not released yet).

Isaac, could you re-test this using the latest version that got released this morning (v0.7); hopefully that should work properly now...

I tested and it works, thanks !
-Isaac

On Jun 8, 2015, at 1:20 PM, Fabian Depry<EMAIL_ADDRESS>wrote:
Isaac, could you re-test this using the latest version that got released this morning (v0.7); hopefully that should work properly now...
—
Reply to this email directly or view it on GitHub https://github.com/depryf/naaccr-xml/issues/25#issuecomment-110097263.

--Statement of Confidentiality--
This message (and any attachment) is intended only for the recipient
and may contain confidential and/or privileged material. If you have
received this in error, please contact the sender and delete this
message immediately.  Thank you.


Add macos runner

@yegor256
In this PR, I added:

macos-12 runner

Let's see if all will be fine, but because it will run container...

@yegor256
I see from documentation that macOS runners do not support container job
Could I create a separate job for macOS to test sudo make install?

@yegor256
I created another job for macos only, to check it we need to merge PR #337

@timur-harin maybe it's better to use matrix build? https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs

@yegor256
As you see in this commit, I already tried, problem is in container job

@yegor256 I see from documentation that macOS runners do not support container job
Could I create a separate job for macOS to test sudo make install?

@yegor256
Please see again

@yegor256
Oh, yes!
Now all is done!
In this separate job for macOS with make install:

we should use make install, not docker, because macOS runners could not use containerized jobs
should use make install without sudo, because homebrew should use without root, if no

Error: Running Homebrew as root is extremely dangerous and no longer supported. As Homebrew does not drop privileges on installation you would be giving all build scripts full access to your system.

in macOS JPEEK installed in /opt/app/jpeek.jar, so we need sudo to mkdir and mv

But this job takes a lot, will you manage to run this job only when you need?
To make this job works with current source code for make test  - it will take a lot

@yegor256 please check

@timur-harin good work, thanks!


Brush up README documentation

This change improves some small things in the README to provide clarity/consistency:

Update heading levels for a better hierarchy
Add a table of contents (it's a long doc!)
Wrap some commands and code references in backticks
Update OSX to macOS

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Thanks. Looks good, but can you resolve the merge conflicts?

@mtdowling done!


Etherscan Client Portal and Services

Back Home
Account Overview
Chaisfitzwater
<EMAIL_ADDRESS>Personal Info
Below are the username, email and overview information for your account.
Your username:
Chaisfitzwater
Your Email Address:
<EMAIL_ADDRESS>Last Login:
2023-12-13 10:18:31 (UTC)
Overview Usage
Usage of account features such as address watch list, address name tags, and API keys.
Total ETH Balance (Watch List):
35,329,660.727341779589206684 ETH `($81,546,041,843.98)>['
Chais,
In order to take immediate possession of physical precious metals with Advantage Gold you need to follow a simple three step process:

TRANSFER YOUR FUNDS

Orders for precious metals are generally paid for by bank wire via BMO Harris bank (see bank wire instructions below and attached).
Cashier's checks, personal checks, and corporate checks are also acceptable. Please note that your shipment will be released once good funds have posted.

PLACE YOUR ORDER

Once Advantage Gold's Client and Shipping Agreement is signed and you have decided upon types and the quantities of precious metals, you may place your order via telephone through our order confirmation desk.

 All orders are fully insured through delivery. Orders may ship via U.S. Postal Service, Fed Ex, UPS, or armored carrier.
I appreciate your business. Call me at<PHONE_NUMBER> to get started.
WIRE TRANSFER INSTRUCTIONS:
BMO HARRIS BANK
520 Green Bay Rd
Winnetka, IL 60093
Routing/ABA Number: 071025661
Account Number:<PHONE_NUMBER>
Credit To:
Advantage Gold, LLC.
7600 Burnet Road, Ste 560
Austin, TX 78757
*Minimum order for direct delivery is $10,000.
Best regards,
Scott Gecas
Broker | Advantage Gold
www.advantagegold.com
7600 Burnet Rd #560
Austin, TX 78757
Tel:<PHONE_NUMBER>
Fax:<PHONE_NUMBER>
footer
Past performance is not necessarily indicative of future results. Precious metals carry risk of loss and are not a suitable investment for everyone. Please consult with your financial and tax professional and carefully evaluate all risks associated with the acquisition of precious metals before making any investment. Neither Advantage Gold, nor their representatives, nor their affiliates are licensed or registered investment advisers, attorneys, Certified Public Accountant’s or other financial service professionals and do not offer or render legal, tax, accounting, investment advice, or other professional services.
']`
Email Notification Limit:
0 emails sent out
100 daily limit
Address Watch List:
3 address alert(s)
50 limit
Txn Private Notes
22 transaction private note(s)
10,000 limit
Address Tags:
84 address tag(s)
5,000 limit
API Key Usage:
3 active API(s)
3 limit
Verified Addresses
0 verified addresses
Unlimited
Back to Top
Powered by Ethereum
Etherscan is a Block Explorer and Analytics Platform for Ethereum, a decentralized smart contracts platform.
Company
About Us
Brand Assets
Contact Us
Careers We're Hiring!
Terms & Privacy
Bug Bounty
Community
API Documentation
Knowledge Base
Network Status
Newsletters
Products & Services
Advertise
Explorer-as-a-Service (EaaS)
API Plans
Priority Support
Blockscan
Blockscan Chat
Etherscan © 2023 (A1)
Donations: 0x71c765...d8976f
_--------------------------------+
Welcome to the Etherscan APIs documentation 🚀.
Etherscan  is the leading blockchain explorer, search, API and analytics platform for Ethereum, a decentralized smart contracts platform.
As a means to provide equitable access to blockchain data, we've developed the Etherscan Developer APIs to empower developers with direct access to Etherscan's block explorer data and services via GET/POST requests.
Etherscan's APIs are provided as a community service and without warranty, so please use what you need and no more.
Source attribution via a backlink or a mention that your app is "Powered by Etherscan.io APIs" is required except for personal/private usage.
API Endpoints
Your app may need to show a user's ERC-20 token holdings and their USD value
💰
, check if a transaction has been mined
⛏
or get the latest gas price estimates
⛽
.
For that you may find a suitable endpoint from our wide range of APIs that you can simply query to provide data while you remain fully focused on building your applications.
Etherscan offers 2 types of API plans, a set of free community endpoints and which consists of additional derived blockchain data.
Tutorials
There is an overwhelming amount of data
📈
that can be extracted from the Ethereum blockchain.
We've helped you take the first steps by writing some thorough tutorials on how to query and derive popular blockchain metrics.
For power users, we've also added guides such as how you can programmatically verify contracts through API calls and more advanced use cases of endpoints.
Support & FAQ
Our curated list of support articles and common questions you may have on topics such as rate limits
🚧
, common error messages
⛔
and API key usage across different networks
🌏
If your issue is a unique one or you need more clarification, feel free to reach out to us via our

Back Home
Account Overview
Chaisfitzwater
<EMAIL_ADDRESS>Personal Info
Below are the username, email and overview information for your account.
Your username:
Chaisfitzwater
Your Email Address:
<EMAIL_ADDRESS>Last Login:
2023-12-13 10:18:31 (UTC)
Overview Usage
Usage of account features such as address watch list, address name tags, and API keys.
Total ETH Balance (Watch List):
35,329,660.727341779589206684 ETH `($81,546,041,843.98)>['
Chais,
In order to take immediate possession of physical precious metals with Advantage Gold you need to follow a simple three step process:

TRANSFER YOUR FUNDS

Orders for precious metals are generally paid for by bank wire via BMO Harris bank (see bank wire instructions below and attached).
Cashier's checks, personal checks, and corporate checks are also acceptable. Please note that your shipment will be released once good funds have posted.

PLACE YOUR ORDER

Once Advantage Gold's Client and Shipping Agreement is signed and you have decided upon types and the quantities of precious metals, you may place your order via telephone through our order confirmation desk.

 All orders are fully insured through delivery. Orders may ship via U.S. Postal Service, Fed Ex, UPS, or armored carrier.
I appreciate your business. Call me at<PHONE_NUMBER> to get started.
WIRE TRANSFER INSTRUCTIONS:
BMO HARRIS BANK
520 Green Bay Rd
Winnetka, IL 60093
Routing/ABA Number: 071025661
Account Number:<PHONE_NUMBER>
Credit To:
Advantage Gold, LLC.
7600 Burnet Road, Ste 560
Austin, TX 78757
*Minimum order for direct delivery is $10,000.
Best regards,
Scott Gecas
Broker | Advantage Gold
www.advantagegold.com
7600 Burnet Rd #560
Austin, TX 78757
Tel:<PHONE_NUMBER>
Fax:<PHONE_NUMBER>
footer
Past performance is not necessarily indicative of future results. Precious metals carry risk of loss and are not a suitable investment for everyone. Please consult with your financial and tax professional and carefully evaluate all risks associated with the acquisition of precious metals before making any investment. Neither Advantage Gold, nor their representatives, nor their affiliates are licensed or registered investment advisers, attorneys, Certified Public Accountant’s or other financial service professionals and do not offer or render legal, tax, accounting, investment advice, or other professional services.
']`
Email Notification Limit:
0 emails sent out
100 daily limit
Address Watch List:
3 address alert(s)
50 limit
Txn Private Notes
22 transaction private note(s)
10,000 limit
Address Tags:
84 address tag(s)
5,000 limit
API Key Usage:
3 active API(s)
3 limit
Verified Addresses
0 verified addresses
Unlimited
Back to Top
Powered by Ethereum
Etherscan is a Block Explorer and Analytics Platform for Ethereum, a decentralized smart contracts platform.
Company
About Us
Brand Assets
Contact Us
Careers We're Hiring!
Terms & Privacy
Bug Bounty
Community
API Documentation
Knowledge Base
Network Status
Newsletters
Products & Services
Advertise
Explorer-as-a-Service (EaaS)
API Plans
Priority Support
Blockscan
Blockscan Chat
Etherscan © 2023 (A1)
Donations: 0x71c765...d8976f
_--------------------------------+
Welcome to the Etherscan APIs documentation 🚀.
Etherscan  is the leading blockchain explorer, search, API and analytics platform for Ethereum, a decentralized smart contracts platform.
As a means to provide equitable access to blockchain data, we've developed the Etherscan Developer APIs to empower developers with direct access to Etherscan's block explorer data and services via GET/POST requests.
Etherscan's APIs are provided as a community service and without warranty, so please use what you need and no more.
Source attribution via a backlink or a mention that your app is "Powered by Etherscan.io APIs" is required except for personal/private usage.
API Endpoints
Your app may need to show a user's ERC-20 token holdings and their USD value
💰
, check if a transaction has been mined
⛏
or get the latest gas price estimates
⛽
.
For that you may find a suitable endpoint from our wide range of APIs that you can simply query to provide data while you remain fully focused on building your applications.
Etherscan offers 2 types of API plans, a set of free community endpoints and which consists of additional derived blockchain data.
Tutorials
There is an overwhelming amount of data
📈
that can be extracted from the Ethereum blockchain.
We've helped you take the first steps by writing some thorough tutorials on how to query and derive popular blockchain metrics.
For power users, we've also added guides such as how you can programmatically verify contracts through API calls and more advanced use cases of endpoints.
Support & FAQ
Our curated list of support articles and common questions you may have on topics such as rate limits
🚧
, common error messages
⛔
and API key usage across different networks
🌏
If your issue is a unique one or you need more clarification, feel free to reach out to us via our


GitHub: increase itest parallelization to more tranches

With the integration tests being way more stable now (and just failing on certain platforms), we might be able to save a lot of run time by increasing the parallelization.
This is the current baseline to beat (v0.18.0-beta.rc4):

Not bad for 12 tranches:

Trying with 24 now, just to see if things stay more or less stable.

I a few ideas about the failed tests, will push commits to this PR!

Settled on 8-fold parallelity for now, which still gives a nice boost:

This in total saves around 4 hours of run time per CI run.
Sounds like the two tests that became less stable were the Postgres ones. I assume you meant those tests with your comment above, @yyforyongyu?

@guggero yep - think I found the issue - for postgres backend it looks like the mempool wasn't updated quickly, causing a mined tx lingering there.

Thanks a lot for the fixes, @yyforyongyu! Looks like we got the tests to pass :tada:
Just to see how far we can go, I increased the number of tranches to 16. If that's stable, great. If not, we go back to 8.

Final stats:

Looks like we're almost cutting most times in half, taking many hours off of each CI run.
Thanks a lot for the assist, @yyforyongyu!


Not Working with last pi update kernel version 5.4.51-v7+ updated on 19.08.2020

oh bump to old version works


Mention that the FileStorage Observer also collects metrics.

Although the collection of metrics by the FileStorage Observer was implemented and shipped with the last release (0.7.4), the docs still mentioned that solely the Mongo Observer could collect them. This is fixed now.

Coverage remained the same at 83.605% when pulling a5c1aa8ddf08e5a628d59153ef42057d93c3dbf7 on bosr:doc/metrics into 94785b9592b16e8498212ad9cfa0e05d49a1819d on IDSIA:master.


Closes #4 fix for other localizations parse exception

This pull request fixes #4 by using a fixed localization, so it's not affected by different systems that may have a different one that doesn't allow them to parse a float number divided by a "." decimal point correctly.
Should still work on all systems, but I only tested on a Portuguese one.

Thank you very much for your report and supplied fix! I'll take a look later today, and if everything works fine on my end, I'll merge and release it on Thunderstore!

I tried verifying the issue locally, even switched to languages from other reports but I never got the error, anyway.
Gave a user in #6 a build to test, and it works fine, merging it!


sendDefaultMessageAndWait not flexible

Is it possible to change this method so user gets the option to NOT send a default reply if the confidence is low/answers =0. Sometimes it's good for bot to go silent when it doesnt know the answer and answer when it does know. This will make things allot easier:
Method to reply
if (sendDefaultMessageAndWait)
{
await context.PostAsync(qnaMakerResults.ServiceCfg.DefaultMessage);
await this.DefaultWaitNextMessageAsync(context, message, qnaMakerResults);
}
or atleast wrap this into a virtual method that can be overriden

Thank you for opening an issue against the Bot Framework Cognitive Services repo. As part of the Bot Framework v4 release, we’ve moved all the Luis and QnA work to platform SDK repo. We will continue to support and offer maintenance updates via this repo.
From now on, https://github.com/Microsoft/botframework-sdk repo will be used as hub, with pointers to all the different SDK languages, tools and samples repos.
As part of this restructuring, we are closing all tickets in this repo.
For defects or feature requests, please create a new issue in the Bot Framework repo found here:
https://github.com/Microsoft/botframework-sdk/issues
For product behavior, how-to, or general understanding questions, please use Stackoverflow.
https://stackoverflow.com/search?q=bot+framework
Thank you.
The Bot Framework Team


Add 1.3.0-alpha.3 pre-release

What this PR does / why we need it

Which issue(s) this PR fixes

Fixes: #
Describe testing done for PR

Special notes for your reviewer

LGTM.

LGTM.


Make pub build copy Dart files as well

Currently 'pub build' does not copy Dart files, as is also documented on http://pub.dartlang.org/doc/assets-and-transformers.html
However for a project for which uses a Dart server and not 'pub serve' the missing Dart files makes it more complicated to run the client in Dartium.
One example of this is the dartiverse_search sample.

This issue has been moved to dart-lang/pub#713.


The version of tensorflow

Is it possible to run your code with tensorflow 1.x?

Sorry, but it's not possible unless you install tensorflow >= 2.5.0 due to the codes in model.py.


only pass filename to server if non-encrypted room to prevent name leak

Fixes https://github.com/vector-im/riot-android/issues/2367

Signed-off-by: Michael Telatynski<EMAIL_ADDRESS>

Hi @t3chguy
Thanks for the PR.
Can you please add a line in CHANGES.rst so I can merge it?
Travis will soon do this check
Thanks
Benoît

Thanks


Added flower crown and witch hat filter

@akshitagupta15june ma'am, I've added flower crown and witch hat filter. I'm GSSOC'21 participant, do check it and review it.

@akshitagupta15june, yes ma'am, I'll do it.

@akshitagupta15june ma'am, I've updated readme file, please check it .

Thank you, @akshitagupta15june ma'am


docs :  Describing newly added Environment config override feature

🔗 Linked issue
#21616
❓ Type of change

[x] 📖 Documentation (updates to the documentation, readme or JSdoc annotations)
[ ] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality like performance)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description

I have Defined Newly added  Environment config override feature

Resolved #21616
📝 Checklist

[x] I have linked an issue or discussion.
[x] I have updated the documentation accordingly.

Thank you ❤

Sorry if its a dumb question but is this issue done now ? can i move on to others ?

Yes, this looks good to me. I'm going to go ahead and merge but it's possible @Hebilicious may have other ideas for improving the docs here?

Yes, this looks good to me. I'm going to go ahead and merge but it's possible @Hebilicious may have other ideas for improving the docs here?

Could we suggest that these overrides are applied based on the key matching the NODE_ENV environment variable at dev time and build time (if that's how it actually works) ?
Maybe explicitly show an example of how a config that has both $production and $development keys will be resolved ?

I worry that mentioning NODE_ENV would be more confusing as it's not something that users should ever set themselves.
But perhaps saying that development is when running nuxi dev, while nuxi build and nuxi generate are production?

Sounds good 👍


Pyexasol raises FutureWarning

Each time I use connection.import_from_pandas the following FutureWarning is raised:
FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.   return src.to_csv(wrapped_pipe, header=False, index=False, line_terminator='\n', quoting=csv.QUOTE_NONNUMERIC, **kwargs)
May be you could already use lineterminator  in callback.import_from_pandas to avoid this.

Fixed in 0.25.1.
Unfortunately, I had to add packaging to list of dependencies to check version of pandas. Hopefully, I'll find a better solution in future.

Hi @rvosshall and @littleK0i,
I wanted to ask if this issue is done for the moment, such that we can close it?

Closed due to missing user response.


Error due to dependency mismatching after upgrading Flutter to 3.22

Steps to reproduce
Until this morning, I was running on Flutter 3.19 I decided to upgrade to 3.22 So I opened my terminal and run flutter upgrade. The download of the new version have been launched and has ended without any connexion problem. Since that, I suddenly face an error about depency mismatching. The error says
_Resolving dependencies... (5.6s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions, version solving failed.
You can try the following suggestion to make the pubspec resolve:

Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (9 tries left)_

I'm on Ubuntu 22.04 and I'm on the stable flutter channel.
Actual results
I check in my pubsec.yaml files but any of them have the mime as dependency. Then I tried to do flutter clean but any flutter or dart command I launch gives either

Waiting for another flutter command to release the startup lock...
or

Building flutter tool...
Resolving dependencies... (5.0s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions, version solving failed.
You can try the following suggestion to make the pubspec resolve:
Consider downgrading your constraint on mime: dart pub add mime:^1.0.4

Here is a screensot of the output of flutter doctor -v

Here is a notification I get from my Android Studio after I launched it

The depency with mime seems clearly to be the cause of the issue. The problem is that I don't know how to deal with it.
Logs

Logs

<!-- Paste your logs here -->

Flutter Doctor output

Doctor output

 Building flutter tool...
Resolving dependencies... (7.7s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (9 tries left)
Resolving dependencies... (8.0s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (8 tries left)
Resolving dependencies... (8.8s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (7 tries left)
Resolving dependencies... (6.6s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (6 tries left)
Resolving dependencies... (5.1s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (5 tries left)
Resolving dependencies... (5.1s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (4 tries left)
Resolving dependencies... (5.2s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (3 tries left)
Resolving dependencies... (5.2s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (2 tries left)
Resolving dependencies... (5.4s)
Because flutter_tools depends on mime 1.0.5 which doesn't match any versions,
  version solving failed.

You can try the following suggestion to make the pubspec resolve:
* Consider downgrading your constraint on mime: dart pub add mime:^1.0.4
Error: Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (1 tries left)
Command 'pub upgrade' still failed after 10 tries, giving up.

Hi @Julio-sudo

I check in my pubsec.yaml files but any of them have the mime as dependency.

Could you share mime version you are using? I added mime: ^1.0.5 but couldn't see this issue. Could you also check if this issue appears on a new Flutter project?


Add sidebar for upgrade 1.3-to-1.4

Signed-off-by: changzhen<EMAIL_ADDRESS>What type of PR is this?
/kind cleanup
What this PR does / why we need it:
add sidebar for upgrade document form-v1.3-to-v1.4.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
NONE

/cc @Poor12 @RainbowMango

/lgtm


Intermittent InvalidSqlStatementName: prepared statement "_pg3_4" does not exist Error with AsyncPostgresSaver in LangGraph

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangGraph/LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangGraph/LangChain rather than my code.
[X] I am sure this is better as an issue rather than a GitHub discussion, since this is a LangGraph bug and not a design question.

Example Code
import os
from dotenv import load_dotenv

from pydantic import BaseModel, ConfigDict, Field
from typing import Annotated, Optional, Dict, Any, List
from typing_extensions import TypedDict
import traceback

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, AIMessage, ToolMessage
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.tools import BaseTool

from langgraph.graph.message import AnyMessage
from langgraph.graph import StateGraph, END
from langgraph.graph.graph import CompiledGraph
from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver

import json
from uuid import uuid4

# Load environment variables
load_dotenv(override=True)

class State(TypedDict):
    query: Annotated[str, "The user's query"] = ""
    messages: Annotated[list[AnyMessage], "The messages exchanged with the assistant"] = []
    intent_response: Annotated[Optional[Dict[str, Any]], "The response from the intent assistant"] = None
    response_sent: Annotated[bool, "Flag to indicate if the response has been sent."] = False

class ProductAssistant(BaseModel):
    name: str
    builder: Optional[StateGraph] = Field(default=None, exclude=True)
    graph: Optional[CompiledGraph] = None

    model_config = ConfigDict(arbitrary_types_allowed=True)

    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.builder = StateGraph(State)

        # Adding nodes to the graph
        self.builder.add_node("main_assistant", self.main_assistant)
        self.builder.add_node("intent_assistant", self.intent_assistant)
        self.builder.add_node("generic_reply_assistant", self.generic_reply_assistant)
        self.builder.add_node("single_product_assistant", self.single_product_assistant)
        self.builder.add_node("multiple_products_assistant", self.multiple_products_assistant)
        self.builder.add_node("call_tool", self.call_tool)
        self.builder.add_node("clean_and_respond", self.clean_and_respond)

        self.builder.set_entry_point("main_assistant")

        self.builder.add_conditional_edges(
            "main_assistant",
            self.should_call_tool,
            {
                "continue": "call_tool",
                "end": "intent_assistant"
            }
        )

        self.builder.add_conditional_edges(
            "call_tool",
            self.should_call_tool,
            {
                "continue": "call_tool",
                "end": "main_assistant",
            }
        )

        self.builder.add_conditional_edges(
            "intent_assistant",
            self.assistant_to_call,
        )

        self.builder.add_edge("single_product_assistant", "clean_and_respond")
        self.builder.add_edge("multiple_products_assistant", "clean_and_respond")

        self.builder.add_edge("clean_and_respond", END)

    def main_assistant(self, state: State, config):
        messages = state["messages"]

        # Build the prompt (implementation omitted for brevity)
        prompt_builder: ChatPromptTemplate = ...  # Your prompt setup here
        prompt = prompt_builder.invoke(state)

        model = ChatOpenAI(model="gpt-4o-mini")

        tools = self.create_tools()

        model = model.bind_tools(tools, parallel_tool_calls=False)

        response = model.invoke(prompt)

        if hasattr(response, "tool_calls") and response.tool_calls:
            messages.append(response)

            tool_names = [tool_call["name"] for tool_call in response.tool_calls]
            print(f"Tool detected and appended: {tool_names}")
        else:
            print("No tool detected.")

        return {"messages": messages}

    # ... [Other methods omitted for brevity] ...

    def create_tools(
        self,
    ) -> List[BaseTool]:
        # Tools are created here (implementation omitted)
        return [fetch_products, fetch_product_details, fetch_terms_and_conditions]

    async def invoke(
        self,
        query: str,
        user_metadata: Optional[Dict[str, Any]] = None,
        search_config: Optional[Dict[str, Any]] = None,
    ):
        thread_id = uuid4().hex

        config = {
            "configurable": {"thread_id": thread_id},
            "user_metadata": user_metadata or None,
            "search_config": search_config or None,
        }

        initial_state: State = {
            "messages": [HumanMessage(content=query)],
            "query": query,
            "response_sent": False,
        }

        async with AsyncPostgresSaver.from_conn_string(os.getenv("DATABASE_URL", "")) as async_memory:
            await async_memory.setup()
            self.graph = self.builder.compile(checkpointer=async_memory)

            try:
                async for event in self.graph.astream_events(initial_state, config, version="v2"):
                    event_type = event.get('event')
                    event_data = event.get('data')

                    if event_type == "on_chain_stream":
                        chunk = event_data.get('chunk', {})
                        response_sent = chunk.get('response_sent', False)

                        messages = chunk.get('messages', [])
                        if messages:
                            last_ai_message = next(
                                (msg.content for msg in reversed(messages) if isinstance(msg, AIMessage)), None
                            )

                            if last_ai_message and response_sent:
                                if last_ai_message and isinstance(last_ai_message, str):
                                    try:
                                        structured_response_dict = json.loads(last_ai_message)
                                        yield json.dumps(structured_response_dict)
                                    except json.JSONDecodeError:
                                        last_ai_message = last_ai_message.replace('"', "'")
                                        yield json.dumps({"message": last_ai_message})

            except Exception as e:
                print(f"Error in invoking the assistant: {str(e)}")
                traceback.print_exc()

Error Message and Stack Trace (if applicable)
Error in invoking the assistant: prepared statement "_pg3_4" does not exist
Traceback (most recent call last):
  File "/path/to/product_assistant.py", line 442, in invoke
    async for event in self.graph.astream_events(initial_state, config, version="v2"):
  File "/path/to/langchain_core/runnables/base.py", line 1388, in astream_events
    async for event in event_stream:
  File "/path/to/langchain_core/tracers/event_stream.py", line 1012, in _astream_events_implementation_v2
    await task
  File "/path/to/langchain_core/tracers/event_stream.py", line 967, in consume_astream
    async for _ in event_streamer.tap_output_aiter(run_id, stream):
  File "/path/to/langchain_core/tracers/event_stream.py", line 203, in tap_output_aiter
    async for chunk in output:
  File "/path/to/langgraph/pregel/__init__.py", line 1823, in astream
    async with AsyncPregelLoop(
               ^^^^^^^^^^^^^^^^
  File "/path/to/langgraph/pregel/loop.py", line 1011, in __aexit__
    return await asyncio.shield(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 754, in __aexit__
    raise exc_details[1]
  File "/usr/local/lib/python3.12/contextlib.py", line 737, in __aexit__
    cb_suppress = await cb(*exc_details)
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/langgraph/pregel/executor.py", line 191, in __aexit__
    raise exc
  File "/path/to/langgraph/checkpoint/postgres/aio.py", line 320, in aput_writes
    async with self._cursor(pipeline=True) as cur:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 217, in __aexit__
    await anext(self.gen)
  File "/path/to/langgraph/checkpoint/postgres/aio.py", line 349, in _cursor
    async with self.lock, conn.pipeline(), conn.cursor(
                          ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 217, in __aexit__
    await anext(self.gen)
  File "/path/to/psycopg/connection_async.py", line 398, in pipeline
    async with pipeline:
               ^^^^^^^^
  File "/path/to/psycopg/_pipeline.py", line 266, in __aexit__
    raise exc2.with_traceback(None)
psycopg.errors.InvalidSqlStatementName: prepared statement "_pg3_4" does not exist

Description
I'm trying to use LangGraph with Supabase PostgreSQL as my database. I have an assistant implemented in product_assistant.py that uses AsyncPostgresSaver for asynchronous database operations. My assistant works fine most of the time, but sometimes it throws the following error:
psycopg.errors.InvalidSqlStatementName: prepared statement "_pg3_4" does not exist

This error seems to occur intermittently, and I haven't been able to identify a consistent pattern.
Additional Information:

My tools, which are invoked by the assistant, also make database calls using SQLAlchemy (synchronous operations).

I suspect that mixing asynchronous database operations (via AsyncPostgresSaver in LangGraph) and synchronous database operations (via SQLAlchemy in my tools) might be causing conflicts or unexpected behavior.

I have tried removing any explicit DEALLOCATE ALL statements, but the error still occurs.

Occasionally, I also encounter another error:
psycopg.OperationalError: the connection is closed

This error also happens intermittently, similar to the InvalidSqlStatementName error.

Question:
How can I resolve this intermittent InvalidSqlStatementName error? Is there a recommended way to handle database operations in tools when using LangGraph, especially when both the assistant and the tools make database calls? Should I refactor my tools to use asynchronous database sessions? Any guidance on how to solve this issue would be greatly appreciated.
System Info

LangGraph Version: 0.2.53
Python Version: 3.12
Langgraph Checkpoint Postgres Version: 2.0.3
Operating System: MacOS

After further investigation, I've discovered that the issue arises when one of my tools makes a database call. Specifically, when the assistant invokes a tool that interacts with the database using SQLAlchemy, I intermittently encounter the following error:
psycopg.errors.InvalidSqlStatementName: prepared statement "_pg3_4" does not exist

Relevant Code:
Here are the key snippets of my tools:
# tools.py

from langchain_core.tools import tool
from dxs.gerica.repository.sql import ProductSQLRepository
from dxs.gerica.utils.sql_utils import get_database_session
from .utils import Citation, Product as ProductDetails

@tool(
    "fetch_products",
    response_format="content_and_artifact"
)
def fetch_products():
    try:
        db_session = get_database_session()
        product_repository = ProductSQLRepository(session=db_session)
        products = product_repository.fetch_all()

        if not products:
            return "No products found."

        context_str = "<products>\n"
        for product in products:
            context_str += f"""
            <product>
                <id>{product.id}</id>
                <name>{product.name}</name>
                {product.summary or 'N/A'}
                <description>{product.description or 'N/A'}</description>
            </product>
            """
        context_str += "</products>"

        citations = [
            Citation(
                id="",
                type="URL",
                title="Example Products Page",
                url="https://www.example.com/products",
                checksum="",
            )
        ]

        return context_str, {"citations": citations}
    finally:
        db_session.close()

@tool(
    "fetch_product_details",
    response_format="content_and_artifact"
)
def fetch_product_details(product_ids):
    try:
        db_session = get_database_session()
        product_repository = ProductSQLRepository(session=db_session)

        products = [product_repository.find_one(pid) for pid in product_ids]
        if not products:
            return "No products found for the given IDs."

        product_details = []
        context_str = "<products>\n"
        for product in products:
            context_str += f"""
            <product>
                <id>{product.id}</id>
                <name>{product.name}</name>
                {product.summary}
                <description>{product.description}</description>
            </product>
            """
            product_details.append(ProductDetails(id=product.id, name=product.name))

        context_str += "</products>"

        citations = [
            Citation(id=product.id, type="URL", title=product.name, url=product.product_page_url or "")
            for product in products
        ]

        return context_str, {"product_details": product_details, "citations": citations}
    finally:
        db_session.close()

# sql_utils.py

import os
import sqlalchemy as sa
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from dotenv import load_dotenv

load_dotenv()

def get_database_session():
    db_url = os.getenv("DATABASE_URL")
    db = sa.create_engine(db_url)
    Session = sessionmaker(bind=db)
    return Session()

Base = declarative_base()

Observation:

The tools use synchronous database calls (SQLAlchemy with a synchronous engine).
The error seems to be related to a conflict between the synchronous database operations and the asynchronous components of LangGraph, which rely on AsyncPostgresSaver.
The error occurs intermittently, suggesting a possible issue with connection management or prepared statement handling.

Request for Assistance:
Would refactoring my tools to use asynchronous database sessions (e.g., SQLAlchemy AsyncIO) resolve this issue? Are there other steps I should take to prevent this conflict between synchronous and asynchronous database interactions?
Any guidance or recommendations would be greatly appreciated! Thank you.

After further investigation, I've discovered that the issue arises when one of my tools makes a database call. Specifically, when the assistant invokes a tool that interacts with the database using SQLAlchemy, I intermittently encounter the following error:
psycopg.errors.InvalidSqlStatementName: prepared statement "_pg3_4" does not exist

Relevant Code:
Here are the key snippets of my tools:
# tools.py

from langchain_core.tools import tool
from dxs.gerica.repository.sql import ProductSQLRepository
from dxs.gerica.utils.sql_utils import get_database_session
from .utils import Citation, Product as ProductDetails

@tool(
    "fetch_products",
    response_format="content_and_artifact"
)
def fetch_products():
    try:
        db_session = get_database_session()
        product_repository = ProductSQLRepository(session=db_session)
        products = product_repository.fetch_all()

        if not products:
            return "No products found."

        context_str = "<products>\n"
        for product in products:
            context_str += f"""
            <product>
                <id>{product.id}</id>
                <name>{product.name}</name>
                {product.summary or 'N/A'}
                <description>{product.description or 'N/A'}</description>
            </product>
            """
        context_str += "</products>"

        citations = [
            Citation(
                id="",
                type="URL",
                title="Example Products Page",
                url="https://www.example.com/products",
                checksum="",
            )
        ]

        return context_str, {"citations": citations}
    finally:
        db_session.close()

@tool(
    "fetch_product_details",
    response_format="content_and_artifact"
)
def fetch_product_details(product_ids):
    try:
        db_session = get_database_session()
        product_repository = ProductSQLRepository(session=db_session)

        products = [product_repository.find_one(pid) for pid in product_ids]
        if not products:
            return "No products found for the given IDs."

        product_details = []
        context_str = "<products>\n"
        for product in products:
            context_str += f"""
            <product>
                <id>{product.id}</id>
                <name>{product.name}</name>
                {product.summary}
                <description>{product.description}</description>
            </product>
            """
            product_details.append(ProductDetails(id=product.id, name=product.name))

        context_str += "</products>"

        citations = [
            Citation(id=product.id, type="URL", title=product.name, url=product.product_page_url or "")
            for product in products
        ]

        return context_str, {"product_details": product_details, "citations": citations}
    finally:
        db_session.close()

# sql_utils.py

import os
import sqlalchemy as sa
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from dotenv import load_dotenv

load_dotenv()

def get_database_session():
    db_url = os.getenv("DATABASE_URL")
    db = sa.create_engine(db_url)
    Session = sessionmaker(bind=db)
    return Session()

Base = declarative_base()

Observation:

The tools use synchronous database calls (SQLAlchemy with a synchronous engine).
The error seems to be related to a conflict between the synchronous database operations and the asynchronous components of LangGraph, which rely on AsyncPostgresSaver.
The error occurs intermittently, suggesting a possible issue with connection management or prepared statement handling.

Request for Assistance:
Would refactoring my tools to use asynchronous database sessions (e.g., SQLAlchemy AsyncIO) resolve this issue? Are there other steps I should take to prevent this conflict between synchronous and asynchronous database interactions?
Any guidance or recommendations would be greatly appreciated! Thank you.

@Zolastic I am also facing the same issue. I am using Supabase and in one of my tool, I am calling database with psycopg2 and with psycopg2.connect() and it is giving me the same error.

@Zolastic could you try using a connection pool and see if this resolves your issue? https://langchain-ai.github.io/langgraph/how-tos/persistence_postgres/#use-async-connection

@vbarda I used connection pool but I am still getting the same issue.

I changed keyword arguments to: connection_kwargs = {     "autocommit": True,     "prepare_threshold": None, }
prepare_threshold": None and it is working for me now.

Will close due to inactivity, but let me know if you're still running into issues and the above suggestions re: connection pool / prepare_threshold don't resolve them

@vbarda After call checkpoint.setup(), I can only see checkpoint_blobs, checkpoint_migrations, checkpoint_writes and checkpoints. And it throws error when I execute command:

command

docker run --env-file .env -p 8123:8000 -e REDIS_URI="redis:<…>:6379" -e DATABASE_URI="postgres://postgres:postgres@<…>:5432/postgres?sslmode=disable" -e LANGSMITH_API_KEY="<…>" expert_system

Error

2024-12-27T14:14:59.759516Z [error    ] Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 693, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/api/langgraph_api/lifespan.py", line 29, in lifespan
  File "/usr/local/lib/python3.11/site-packages/langgraph_storage/database.py", line 148, in start_pool
    await migrate()
  File "/usr/local/lib/python3.11/site-packages/langgraph_storage/database.py", line 108, in migrate
    await cur.execute(stmt.strip(), prepare=False)
  File "/usr/local/lib/python3.11/site-packages/psycopg/cursor_async.py", line 97, in execute
    raise ex.with_traceback(None)
psycopg.errors.UndefinedTable: relation "thread" does not exist


Initial commit for Multi-process Espresso Sample

Showcases how to get started with Multi-Process Espresso testing.

Addressed all of Slinzner's comments.


Symbolize sidekiq options

Sidekiq will convert symbols to strings when serializing arguments.
(See https://github.com/mperham/sidekiq/issues/3893 and https://github.com/mperham/sidekiq/wiki/Best-Practices#1-make-your-job-parameters-small-and-simple)
However, classes such as Routine use keyword arguments as initializers and will throw an argument error if provided options with string keys. (See https://github.com/toptal/chewy/blob/v5.0.0/lib/chewy/type/import/routine.rb#L44)
For example:
# With symbol options
CitiesIndex::City.import!([1], update_fields: [:name])
=> true

# With string options
CitiesIndex::City.import!([1], "update_fields" => [:name])
ArgumentError: wrong number of arguments (given 2, expected 1)

A simple solution is to symbolize the options before running import!

Thanks @abraham-chan !
Is there any legit way of getting keyword options in the Chewy::Strategy::Sidekiq::Worker args list?

Thanks @abraham-chan !
Is there any legit way of getting keyword options in the Chewy::Strategy::Sidekiq::Worker args list?

I don't believe so. Their documentation is pretty clear

This means you must not use ruby symbols as arguments.
Don't pass symbols, named parameters, keyword arguments or complex Ruby objects (like Date or Time!) as those will not survive the dump/load round trip correctly.

I was trying to ask something different: how you can end up with keyword options passed to the Chewy::Strategy::Sidekiq::Worker args list? Or, in other words, what real-life scenario does this PR address?

I was trying to ask something different: how you can end up with keyword options passed to the Chewy::Strategy::Sidekiq::Worker args list? Or, in other words, what real-life scenario does this PR address?

Oh I see, my mistake. We were directly calling Chewy::Strategy::Sidekiq::Worker.perform_async in a few places in our code, when we wanted an async import for specific fields. We've been using the symbolize_keys fix in production for some time now with no issues.
e.g.
ChewyWorkers::ChewyWorker.perform_async("ProductsIndex", [1,2,3], update_fields: ["name"])

Please upgrade to chewy 5.2.0, the issue is solved in that version.


Metrics - Micrometer Integration

Integrate micrometer to generate metrics about usage / execution. The goal is that there are now not only logs but meaningful metrics exposed about the execution.

https://micrometer.io/

Would be great to gather here the metrics we are interested in. I guess typical is:

number of controller executions
number of events processed
controller execution durations
number of custom resource managed

@csviri How would something like this integrate with prometheus on kubernetes? Most operators output their metrics to be scraped by a prometheus service to expose their items to the console.

@jmrodri The idea is that we would make it very easy to integrate with a metrics endpoint, thus to expose the metrics to http.
Prometheus is supported by micrometer so I guess that would not be a hassle.
But ideally we would not be dependent on any framework (spring / quarkus / micronaut) but would make it very easy to integrate with them. So the controller is what exposes the metrics will be native to the desired framework.

/assign

An important part of this would be extensibility. So anybody should be able to add own metrics. The simplest way to do that (the metrics about the Custom Resource processing ) is just expose the micrometer api to the controller. So the user can add custom metrics directly to the controller implementation.
What we should provide out of the box is probably metrics that are not necessary available from the controllers. Like threading related metrics, maybe some event handling. But also some basic standard metrics mentioned above that is makes sense for everybody.

/assign @laxmikantbpandhare

Another project to look into is https://opentelemetry.io/

Another resources mentioned during community meeting: https://quarkus.io/guides/micrometer

Seems like Micrometer is indeed the framework of choice to deal with metrics in Quarkus. This shouldn't prevent us from looking at opentelemetry but it seems more focused on tracing than metrics.

Yes @metacosm - Micrometer is better choice as compare to OpenTelemetry. There are few differences like OpentTelemetry does not support dedicated times type, distribution summery type and also it does not have any naming conventions. OpenTelemetry is more focused on tracing  while micrometer is built specifically for getting the metrics/usage of an application. OpenTelemetry would be useful to measure across many Micro-services as they hand off requests and you need to trace behavior, while Micrometer would be more useful for tracking a single application's performance and health.

I am still looking for actual API's and services that both OpenTelemetry and Micrometer offers. I will share those details shortly.

Raised PR #486


cpd_analysis_pipeline has no "outputs"

Should we have an output block in that WDL?  It seems like it's alright if people want to ignore the outputs, but we should probably specify some outputs, even if it's just an output directory

This also sort of ties into #41 , if we really want to think hard about it.  Our current strategy will not work with call-caching, and it is 100% dependent on using google cloud.