Pylance v2023.10.50 is incompatible with Python Extension v2020.8.109390

Environment data

Language Server version: v2023.10.50
OS and version: Windows 11 Pro 22H2 22621.2428
Python version (& distribution if applicable, e.g. Anaconda): 3.5.4

Code Snippet

XXX

Repro Steps

Since I‘m using Python 3.5.4, I switched Python extension to v2020.8.109390 so that I can use the debugger. In previous versions everything works fine until v2023.10.50. I got Request initialize failed with message: You may install and use any number of copies of the software only with Microsoft Visual Studio ..., but I actually did not do anything it listed above.

Switch Python extension to v2020.8.109390
Use latest Pylance
Open a Python file, you can see the Python extension shows 2 error messages
Switch Pylance to v2023.10.40, or use latest Python extension
It works

Expected behavior
Python extension works fine
Actual behavior

Logs

Output from Python Language Server:
[Info  - 11:20:59 AM] (5568) Pylance language server 2023.10.50 (pyright cfe56bfb) starting
[Info  - 11:20:59 AM] (5568) Server root directory: C:\Users\Yolo\.vscode\extensions\ms-python.vscode-pylance-2023.10.50\dist
[Error - 11:20:59 AM] Server initialization failed.
  Message: Request initialize failed with message: You may install and use any number of copies of the software only with Microsoft Visual Studio, Visual Studio for Mac, Visual Studio Code, Azure DevOps, Team Foundation Server, and successor Microsoft products and services (collectively, the “Visual Studio Products and Services”) to develop and test your applications. The software is licensed, not sold. This agreement only gives you some rights to use the software. Microsoft reserves all other rights. You may not: work around any technical limitations in the software that only allow you to use it in certain ways; reverse engineer, decompile or disassemble the software, or otherwise attempt to derive the source code for the software, except and to the extent required by third party licensing terms governing use of certain open source components that may be included in the software; remove, minimize, block, or modify any notices of Microsoft or its suppliers in the software; use the software in any way that is against the law or to create or propagate malware; or share, publish, distribute, or lease the software (except for any distributable code, subject to the terms above), provide the software as a stand-alone offering for others to use, or transfer the software or this agreement to any third party.
  Code: -32603
[Error - 11:20:59 AM] Starting client failed
  Message: Request initialize failed with message: You may install and use any number of copies of the software only with Microsoft Visual Studio, Visual Studio for Mac, Visual Studio Code, Azure DevOps, Team Foundation Server, and successor Microsoft products and services (collectively, the “Visual Studio Products and Services”) to develop and test your applications. The software is licensed, not sold. This agreement only gives you some rights to use the software. Microsoft reserves all other rights. You may not: work around any technical limitations in the software that only allow you to use it in certain ways; reverse engineer, decompile or disassemble the software, or otherwise attempt to derive the source code for the software, except and to the extent required by third party licensing terms governing use of certain open source components that may be included in the software; remove, minimize, block, or modify any notices of Microsoft or its suppliers in the software; use the software in any way that is against the law or to create or propagate malware; or share, publish, distribute, or lease the software (except for any distributable code, subject to the terms above), provide the software as a stand-alone offering for others to use, or transfer the software or this agreement to any third party.
  Code: -32603

Same issue,  Have to roll back to v2023.10.40

Just received the same message!

Same. Broken.

I have seemingly this problem too, on first glance, but I have a nuance to add:
the versions are not the same, and rolling back didn't solve it.
So probably not the same problem in the end, but I'm posting it here anyway, in case it helps.
Some further investigation lead me to think that something is broken about the specific python version I use. Indeed, changing the python interpreter fixes the problem.

Same issue for me,
I have Python extension v2022.16.1 and Python 3.9.18.
It works if I roll back to v2023.10.40, as suggested here.

Not eve 10.40 works. It's all broken.

I am using Python v2022.8.0 extension and Python 3.16.15 in devcontainer.
Today after updating Pylance I have the same big error. Pylance doesn't work at all.

Absolutely perfectly it was helped to me downgrading to Pylance  2023.10.40 version. This is very good Pylance to me

Rolling Pylance language server back to 2023.10.40 solved this for me. 99% of you probably know this already, but I had to muck around Google to figure out how to roll back to previous version, so if anyone needs it, here's the quick how to:

Open VS Code.
Click the Extensions icon in the left-hand pane (or Ctrl+Shift+X).
Scroll to the Pylance extension.
Click the tiny gear icon next to the Pylance extension.
Choose "Install another version …" from the context menu.
Choose 2023.10.40 (or whatever version works for you).

Hey everyone. Unfortunately it's not feasible for us to maintain Pylance compatible with older versions of the Python extension. However, if the main reason why you're using an old version of the Python extension is support for debugging Python 2.7 or Python 3.5/3.6, there may be a better workaround so you can keep using the latest Pylance and Python extensions.
We have a new Python Debugger extension that we are shipping separately so people can keep using the latest versions of the Python extension while having support for debugging older versions of Python. Specifically, the Python Debugger extension versions  2023.1.* support debugging for Python versions 3.6 and older.
This way you can continue updating the Pylance and Python extensions and only use this one older version of the Python Debugger extension.

I have similar problem with v2023.18.0 of Python Extension as well. It does not happen right away. It just crashed randomly when I am using Jupyter notebook.

Same here with latest extension on Python (v2023.18.0) and Pylance (v2023.10.50). Changing to previous Pylance (v2023.10.40) worked for me aswell)
Error:
2023-10-27 20:03:33.046 [info] [Error - 8:03:33 PM] Restarting server failed
2023-10-27 20:03:33.046 [info]   Message: Request initialize failed with message: You may install and use any number of copies of the software only with Microsoft Visual Studio, Visual Studio for Mac, Visual Studio Code, Azure DevOps, Team Foundation Server, and successor Microsoft products and services (collectively, the “Visual Studio Products and Services”) to develop and test your applications. The software is licensed, not sold. This agreement only gives you some rights to use the software. Microsoft reserves all other rights. You may not: work around any technical limitations in the software that only allow you to use it in certain ways; reverse engineer, decompile or disassemble the software, or otherwise attempt to derive the source code for the software, except and to the extent required by third party licensing terms governing use of certain open source components that may be included in the software; remove, minimize, block, or modify any notices of Microsoft or its suppliers in the software; use the software in any way that is against the law or to create or propagate malware; or share, publish, distribute, or lease the software (except for any distributable code, subject to the terms above), provide the software as a stand-alone offering for others to use, or transfer the software or this agreement to any third party.
Code: -32603

Hey everyone. Unfortunately it's not feasible for us to maintain Pylance compatible with older versions of the Python extension. However, if the main reason why you're using an old version of the Python extension is support for debugging Python 2.7 or Python 3.5/3.6, there may be a better workaround so you can keep using the latest Pylance and Python extensions.
We have a new Python Debugger extension that we are shipping separately so people can keep using the latest versions of the Python extension while having support for debugging older versions of Python. Specifically, the Python Debugger extension versions 2023.1.* support debugging for Python versions 3.6 and older.
This way you can continue updating the Pylance and Python extensions and only use this one older version of the Python Debugger extension.

I am working with Python 3.10 and it's still giving me the same error rolling back to a previous version fixed it.

Hey everyone. Unfortunately it's not feasible for us to maintain Pylance compatible with older versions of the Python extension. However, if the main reason why you're using an old version of the Python extension is support for debugging Python 2.7 or Python 3.5/3.6, there may be a better workaround so you can keep using the latest Pylance and Python extensions.
We have a new Python Debugger extension that we are shipping separately so people can keep using the latest versions of the Python extension while having support for debugging older versions of Python. Specifically, the Python Debugger extension versions 2023.1.* support debugging for Python versions 3.6 and older.
This way you can continue updating the Pylance and Python extensions and only use this one older version of the Python Debugger extension.

The issue is happening even when using the latest versions of both Pylance and Python extensions.

For everybody that's reproing the issue, can you state:

VS Code Version?
Local or remote?
Python extension version?

Thanks. We can't seem to reproduce it in house.
Note: This error is expected if you're not actually using VS code (like VS codium or Cursor). It is NOT expected if you're using VS code through SSH/WSL/Dev Container. Those should work.

This issue started today for me.
VS Code 1.83.1
Remote (WSL on same machine)
Python extension version v2023.18.0

Can you try 2023.10.53? I wonder if it's related to this issue: https://github.com/microsoft/vscode/issues/196740. Our best guess on the other issue is that the crashes in the Pylance server are starving the VS code extension host, preventing other things from working. It might be preventing the new message we added that verifies we're running inside of VS code.

If you see this in the Python Language Server output:
Debug Failure. Unable to find notebook URI for vscode-notebook-cell:

2023.10.53 might be the fix for the problem. It fixed that error.

For everybody that's reproing the issue, can you state:

VS Code Version?
Local or remote?
Python extension version?

Thanks. We can't seem to reproduce it in house.
Note: This error is expected if you're not actually using VS code (like VS codium or Cursor). It is NOT expected if you're using VS code through SSH/WSL/Dev Container. Those should work.

MacOS Sonoma 14.0 VS Code: 1.83.1 Pylance: v2023.10.50 Python: 3.11.5
Local workspace. Virtual Python environment. Using JupyterNotebooks. Rolling back to Pylance v2023.10.40 seems to have fixed the problem. Issue happened in general use, switching between notebooks, scrolling. I haven't been actively executing code all that often.

Sorry @csf90-uk what's your Python extension version? You listed your Python version, but the Python extension needs to be a certain version for Pylance to work too.

Vs Code version: 1.83.1
Remote: SSH
python 3.11.3

For everybody that's reproing the issue, can you state:

VS Code Version?
Local or remote?
Python extension version?

Thanks. We can't seem to reproduce it in house.
Note: This error is expected if you're not actually using VS code (like VS codium or Cursor). It is NOT expected if you're using VS code through SSH/WSL/Dev Container. Those should work.

VS Code Version: 1.83.1
Remote: SSH
Python extension version: v2023.18.0
Since my problem mainly happened during use of notebook, Jupyter extension version: v2023.9.1102792234. This could purely depend on the workload at the time is mainly using notebook. The symptoms for me is basically, when typing/scrolling through the code, pylance died with the message, also brought down the Jupyter kernel as well, no way to restart the kernel. Needs to reload the window to restart.

Vs Code version: 1.83.1 Remote: SSH python 3.11.3

Sorry the Python extension. It should be something like this:

For everybody that's reproing the issue, can you state:

VS Code Version?
Local or remote?
Python extension version?

Thanks. We can't seem to reproduce it in house.
Note: This error is expected if you're not actually using VS code (like VS codium or Cursor). It is NOT expected if you're using VS code through SSH/WSL/Dev Container. Those should work.

MacOS Sonoma 14.0 VS Code: 1.83.1 Pylance: v2023.10.50 Python: 3.11.5
Local workspace. Virtual Python environment. Using JupyterNotebooks. Rolling back to Pylance v2023.10.40 seems to have fixed the problem. Issue happened in general use, switching between notebooks, scrolling. I haven't been actively executing code all that often.

Sorry @csf90-uk what's your Python extension version? You listed your Python version, but the Python extension needs to be a certain version for Pylance to work too.

Apologies. v2023.18.0

@csf90-uk @Tma2333-Scale and for those who have python extension 2023.6.0 or a newer version installed, were you seeing the exact same message You may install and use any number of copies of the software only with Microsoft Visual Studio..., or did you experience a different Pylance crash?

Yes, the same error.
On Wed, 1 Nov 2023 at 06:30, Stella @.***> wrote:

@csf90-uk https://github.com/csf90-uk @Tma2333-Scale
https://github.com/Tma2333-Scale and for those who have python
extension 2023.6.0 or a newer version installed, were you seeing the exact
same message You may install and use any number of copies of the software
only with Microsoft Visual Studio..., or did you experience a different
Pylance crash?
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/pylance-release/issues/5019#issuecomment-1788481128,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXMSZMAMFCOG7F4WIBDVVTLYCHT7PAVCNFSM6AAAAAA6SFTW7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBYGQ4DCMJSHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

For everybody that's reproing the issue, can you state:

VS Code Version?
Local or remote?
Python extension version?

Thanks. We can't seem to reproduce it in house.
Note: This error is expected if you're not actually using VS code (like VS codium or Cursor). It is NOT expected if you're using VS code through SSH/WSL/Dev Container. Those should work.

VS Code Version: 1.83.1
Remote: SSH
Python extension version: v2023.18.0

Same here, only downgrading pylance worked.

can we get some screenshots of extensions tab and output window when the issue is happening?

currently, we can't seem to repro and none of report here provide some actionable info for us to dig in.
those screenshots will at least let us know whether we are using right env to repro the issue.

Most recent VSCode, pylance, etc. For me, it's breaking on the conda environment provided by ArcGIS Pro 3.1.3, which uses python v3.9.16. Worked "fine" (although pylance can't produce intellisense for all submodules of arcpy even on the best days) last time I used this env a few months ago. Going back a few versions of pylance fixed it for me.

@quillaja I am sorry to hear that the latest pylance didn't work as expected but we will need more info to find out why that happened. providing screenshots as above would help us to find out root issue!
thank you

can you show us not only pylance version but python extension version as well? as long as we know, the failure should only happen when pylance extension is used with very old python extension we no longer support.
that said, your screen shows different issue than this issue - Request initialize failed with message: You may install and use any number of copies of ...
you probably want to open a new issue so we can take a look rather than mixing with this issue.
thank you.

@quillaja I think your particular issue should be fixed by the latest pylance prerelease (2023.10.53)

feedback my timeline:

2023.10.50 bad
(found this issue, same Code: -32603)
2023.10.53 bad
2023.10.40 good

I'm having the same issue.
All version info is in the screenshot.

@neljo8m thank you very much for the screenshot. is your dev environment vscode, local, mac and this happen with any python file?

@liuxhit can you share the screenshot?

@neljo8m thank you very much for the screenshot. is your dev environment vscode, local, mac and this happen with any python file?

Yes, I'm using VS Code 1.83.1, local on Mac OS 13.6.1.
It is not specific to any particular file.  And I have not noticed any pattern when it happens.
I did notice some folks here mentioned that it was related to Jupyter.
So it is interesting, this time, it did happen just after I ran a few lines of code in Jupyter, using Py comments.
Like this:
# %%
# run some python code here
v = "test"
print( f"{v=}" )
# %%

When I pressed <Shift+Enter> to run that in Jupyter, it ran fine.
But shortly after that, I received the Pylance error.
So maybe it is related to Jupyter in some way.  Or maybe Jupyter just starts the problem.
I have the Jupyter plugin v2023.10.1003070148 (Microsoft)

@neljo8m there is a separate jupyter related issue that is fixed in pylance 2023.10.53. without log, hard to know whether which issue you are hitting. this issue is specifically about Initialization failure issue due to using the latest pylance with python extension we no longer support.

@neljo8m there is a separate jupyter related issue that is fixed in pylance 2023.10.53. without log, hard to know whether which issue you are hitting. this issue is specifically about Initialization failure issue due to using the latest pylance with python extension or jupyter extension we no longer support.

You see the version of the Python extension I am using.  So it looks like that does not apply to me.

@neljo8m

You see the version of the Python extension I am using. So it looks like that does not apply to me.

are you referring jupyter issue or intialization failed issue?

Sorry, I was not clear.
I was referring to your comment about the "issue due to using the latest
pylance with python extension or jupyter extension we no longer support".
Since I am using the latest versions, that issue does not apply.

@neljo8m

You see the version of the Python extension I am using. So it looks like that does not apply to me.

are you referring jupyter issue or intialization failed issue?

Sorry, I was not clear.
I was referring to your comment about the "issue due to using the latest
pylance with python extension or jupyter extension we no longer support".
Since I am using the latest versions, that issue does not apply.

This issue has been fixed in prerelease version 2023.11.21, which we've just released. You can find the changelog here: CHANGELOG.md

@bschnurr Thank you for announcing.

This issue has been fixed in prerelease version 2023.11.21, which we've just released.

You mean version 2023.11.12?

You mean version 2023.11.12?

Yes, thanks. Fixed above.

This issue is still (again?) happening on version 2024.12.3 of the Python extension.

@ljnsn, questions for you:

Why are you using a 9-month old version of the Python extension?
What version of Pylance are you trying to use?
Pylance is only supported in official builds of VS Code. Are you using one?

We don't maintain a mapping of compatible historical versions of the Pylance and Python extensions. My recommendation would be to update to the latest of both.

@debonte thanks for the quick response!

That's the version that got installed, isn't it the latest one? https://github.com/microsoft/vscode-python/releases/tag/v2024.12.3 released four days ago?
Pylance is 2024.8.1
It's the build provided by my package manager, so not the official one. vscode is version 1.91.1

We don't maintain a mapping of compatible historical versions of the Pylance and Python extensions. My recommendation would be to update to the latest of both.

I understand, but the versions I'm using are the latest ones, aren't they?

Oh, I'm sorry. I forgot that the Python extension has a different versioning approach than us. I was interpreting xxxx.12.3 as a December release and didn't even look at the year.

It's the build provided by my package manager, so not the official one. vscode is version 1.91.1

Is this VS Code OSS then? Pylance is only supported in the official builds. Try downloading and installing from the VS Code website and I think you'll find that it works.

Yes, it's a version of VS Code OSS. I previously never had any issues with Pylance/Python on this build, even though it's not officially supported. I'll wait for an update from the repo maintainer and see if it works with 1.92.x, otherwise I'll have to keep it updated myself.


Invalid version specifiers in requirements.txt

Client Operating System
Linux
Python version
3.7.9
Problem description
Package version syntax without delimiting commas is used in requirements.txt.  For example here.  I would expect that to be beautifulsoup4>=4.7.0,<4.8.0.
When installing redshift_connector via pip (within a conda environment), I am seeing the following error:
InvalidVersionSpec: Invalid version '4.7.0<4.8.0': invalid character(s)

which can be traced back to beautifulsoup4>=4.7.0<4.8.0 in the requirements.txt file.
Based on PEP 440, delimiting commas are a required part of the version clause.  I think this is likely nuanced as I am able to install redshift_connector via pip in some cases (for example on my local Windows machine) but hit this error when installing in a cloud Linux environment.  Is there a reason not to use separating commas?  Is it a supported syntax that I am not aware of?

I need to correct the above.  I am running in an error when exporting a conda environment that has redshift_connector installed.  I am able to create the conda environment and pip install redshift_connector into it without issue.
conda env export -n my-env

gives
InvalidVersionSpec: Invalid version '4.7.0<4.8.0': invalid character(s)

Hey @hunter-moeller ,
Thank you for raising this issue. The team will take a look into reproducing and resolving this issue and provide an update shortly.

Hey @hunter-moeller ,
When taking the steps you described:

conda create --name myenv
conda activate myenv
conda install pip
pip install redshift_connector
conda env export -n myenv

I was able to reproduce this error. We will be including a fix for this in our next release of redshift_connector, which is scheduled to occur next week.
As a sidenote -- if redshift_connector is installed via conda, the environment can be exported without issue:

conda create --name myenv1 redshift_connector
conda env export -n myenv1

note: this was only tested using redshift_connector-2.0.874
Thanks again for bringing this to our attention! :)

Awesome, thank you for investigating this.

Fix for this issue provided in v2.0.875


Evaluation: Track evaluator usage through user agent

Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:

[ ] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[ ] I have read the contribution guidelines.

General Guidelines and Best Practices

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

For the time being, Multi-modal is not covered in this PR, as they may be integrated into content safety evaluators later on.
Feature 3594503: Merge multi modal evaluators into regular evaluators


Fix PDF guidelines in downloadable zip

The https://github.com/music-encoding/music-encoding/archive/v3.0.0.zip contains the MEI_Guidelines_2013_v2.1.1.pdf in the guidelines directory rather than the MEI_Guidelines_v3.0.0.pdf available at http://music-encoding.org/downloads/latest-release/.

@kepper I believe this is up to you to generate the new guidelines PDF.

@ahankinson I'm happy to do whatever is necessary, but I don't know what to do in my clone to downstream it correctly in the official repo. As @musicus pointed out, the correct PDF is available already, living at http://www.music-encoding.org/docs/MEI_Guidelines_v3.0.0.pdf.

Am 10.11.2016 um 05:14 schrieb Andrew Hankinson<EMAIL_ADDRESS>@kepper I believe this is up to you to generate the new guidelines PDF.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


Integration as cli

Need to be able to run from command line. Make sure that stdin is tested.

Binding into python directly.


AfterStep hook doesn't run if the step fails

with behat 2.50 + mink-extension I had an afterStep hook which takes an screenshot when a step fails, with behat 3.0 that hook does not run when the step fails, I modified one of the hooks.feature scenario adding an afterStep hook and making a step fail to check and the same result (the hook does not run), it is that the expected behaviour?

This bug is happening again on the 3.1.1 version


Improvement - Show Error log from authenticate service

While using auth_web.py I was getting 500 error. It would have been good if logs of error are shown on screen.
Adding these lines put errors on browser.
conf = {'global':{'request.show_tracebacks':True}}
cherrypy.config.update(conf)
Can we add these lines to code.

Could you make a PR?

sure !


How to Add Configuration Parameters When Starting a Docker Container

How can I add configuration parameters when starting a Docker container, similar to go2rtc -c log.format=text -c rtsp.listen=''? If possible, could you provide the complete docker run command?
Looking forward to your reply

This is a general question about docker usage. Not directly related to this project. You need to read the docker documentation.

Here's how I'm doing it in my docker-compose.yml file, in order to avoid a bind mount for the config file, so that I can throw it into docker swarm:
services:
  go2rtc:
    image: art.t3.daimlertruck.com/godap-docker-virtual/alexxit/go2rtc:latest
    privileged: true
    environment:
      - TZ=America/Los_Angeles
#    network_mode: host
    ports:
      - 1984:1984
      - 8554:8554
      - 8555:8555
#    volumes:
#      - "./go2rtc.yaml:/config/go2rtc.yaml"
# Info on constructing the command statement instead of using
# the go2rtc.yaml config file and bind mount can be found here
# https://github.com/AlexxIT/go2rtc/blob/master/internal/app/README.md
    command: >
      go2rtc
        -config api.listen='<IP_ADDRESS>:1984'
        -config streams.bambu_x1e='rtsps://${X1E_USR}:${X1E_ACCESS_CODE}@${X1E_IP_ADDR}/streaming/live/1'
        -config log.level='debug'


Streetmix doesn't load when uBlock Origin is enabled

From user feedback:

Just noticed that the page doesn't load when using uBlock Origin. Since you don't have any ads anyways, it would be helpful to not need to use incognito mode to view the site.

TODO:

[ ] Verify this is a bug
[ ] If so, fix it

Loads fine here (http://streetmix.net/-/411697), using default settings. Tried on Chrome 54 and Firefox Nightly 51.

Yeah; I've got uBlock Origin enabled as well, and this has never seemed to be a problem for me.
This was the original user agent string from the user's report, by the way: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
Also, @gorhill, did you have a filter for any public repos that mention uBlock Origin or something? Thanks for dropping by. The plugin is fantastic!
Closing now - no bug found.

did you have a filter for any public repos that mention uBlock Origin or something?

I poll Twitter once in a while for "uBlock" to find out if there are issues -- there was a tweet in there linking here.

Ah - found it! https://twitter.com/civicissues/status/762260443968581632
Super helpful bot.


ISV-5245 - update original chart to use deployments instead of cron jobs

To bring OpenShift template closer to charts/trustification this PR is updating original Helm chart to use deployment instead of cron job for bombastic-collector and vexination-collector.

Changes look good to me. I'll defer to Max and Marco for more thoughts on how to proceed with this

Closing as for the long-term solution we decided to deploy directly with the TPA Helm chart.


Remove a fixme

Just saw the new release of SmolStr and was reminded of this FIXME I added :)

bors r+
Thanks!
On Wednesday, 10 July 2019, Laurențiu Nicola<EMAIL_ADDRESS>wrote:

@lnicola approved this pull request.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rust-analyzer/rust-analyzer/pull/1518?email_source=notifications&email_token=AANB3MYZXJAYZFD27OVDECDP6X4HVA5CNFSM4H7QLYZKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB6A5LNA#pullrequestreview-260167092,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANB3M32LZIFNNG3V6BH7EDP6X4HVANCNFSM4H7QLYZA
.


Implement webhook endpoint for garo-server

Garo server needs a /webhook endpoint which can be consumed by github.com
The webhook endpoint should be protected by a configurable secret to prevent abuse of this API.
Optionally IP whitelisting can be implemented. To prevent requests to this endpoint from different IPs.
The check_run webhook should enable us to check if a build failed due to no available runner. If this occurs Garo-server needs to track this so the agents can retrieve for which repositories a runner has to be deployed and as well which runs need to be retriggered.

@npalm could you add the involved curl requests to retrieve the run-failure?

I would suggest we implement a webhook and dispatch event to a command handler for specific events. to start with we only need an event listener for the check_run event.


Elaborate all the ROS_INFO

Do this in all task handlers and motion library.

Please explain...

Where ever we are printing "pwm send to arduino" or "trusters stopped" add which thrusters or which node has stopped the thrusters.

There should be no cout or printf.

closed by #176


Reviewing StreamResource and StreamDatum from the storage, access perspective

During the discussions that developed StreamResource and StreamDatum documents, the "producer" side was being built, but much of the "consumer" side remained to be developed. I think we agreed to stay open to adjustments when the time came to build out the consumer side. I hope that window is still open.
Here's a StreamResource from DLS I22, courtesy of @DiamondJoseph:
  {
    "name": "stream_resource",
    "doc": {
      "uid": "158167d9-796b-4e12-afff-5b3dd903815c",
      "data_key": "waxs-sum",
      "spec": "AD_HDF5_SWMR_SLICE",
      "root": "/dls/i22/data/2023/cm33873-5",
      "resource_path": "i22-723667-waxs-hdf.h5",
      "resource_kwargs": {
        "path": "/entry/instrument/NDAttributes/StatsTotal",
        "multiplier": 1,
        "timestamps": "/entry/instrument/NDAttributes/NDArrayTimeStamp"
      },
      "path_semantics": "posix",
      "run_start": "617184e5-5e24-40b1-80f7-0056d51e44f8"
    }
  }

I can see nothing to change about these:

uid, unique key for the document
data_key, tells us which field (column) in the Event Stream this maps to
run_start, tells us which BlueskyRun this belongs to

I might suggest re-evaluating that names of:

spec, a key in a registry that tells us which reader ("handler") to use
resource_kwargs, additional configuration for the reader

In Tiled, we use MIME types, including standard ones such as image/tiff and custom ones named like application/x-hdf5-smwr-slice, in the role of spec. Maybe it's worth considering whether we want to use mimtype here, as it is a recognized standard way to spell, "This is a format, which tells you which I/O code to read it."
Taking a broader-than Python look at the world, resource_kwargs (emphasis on "kwargs") is a bit of a Python-ism. These are JSON-serializable parameters that are needed by the code that reads the data. This code does not necessary have to be in Python. In Tiled, the analogous entity is simply called parameters.
The existing names aren't doing any harm, so we have to weigh the pain of change against the marginal benefit of maybe-better names. If we leave them as is, I think that would be fine. Just worth considering.
The way we spell the location of the data seems more problematic:

root, an absolute filepath
resource_path, a relative filepath (sometimes filename)
path_semantics, a two-member enum that is win or posix

I think we should consider replacing all of this with a URI, like file://localhost/dls/i22/data/2023/cm33873-5/i22-723667-waxs-hdf.h5".

We foresee data being available in protocols other than file:, such as s3:.
The partitioning of the path between root and resource_path is a guess about what parts of the path will be stable over time and what parts may change as the data moves. Different people make different guesses, and at NSLS-II this has been a mess.
IIRC, path_semantics is completely vestigial, based on a misunderstanding that Windows required backslashes in paths. In fact, anything newer than Windows 95 is fine with forward slashes.

In Tiled, the same logical resource may be available in multiple places. This is expressed as a one-to-many relation between logical datasets and URIs (effectively). If the StreamResource just had a URI, this would be simpler and more future-proof.
Here is a StreamDatum:
  {
    "name": "stream_datum",
    "doc": {
      "stream_resource": "158167d9-796b-4e12-afff-5b3dd903815c",
      "uid": "158167d9-796b-4e12-afff-5b3dd903815c/2",
      "seq_nums": {
        "start": 6,
        "stop": 7
      },
      "indices": {
        "start": 5,
        "stop": 6
      },
      "descriptor": "8ff32942-9b21-4542-aee1-16c04291950d"
    }
  }

The descriptor is on StreamDatum instead of on StreamResource with data_key. I think it's worth revisiting whether we really really need that, because life would be simplify if the descriptor and data_key were together on the same document.

I'm agnostic to the key name changes.
Adding the file: vs s3: vs ... should be safe to do in a back-compatible way if we assume that anything with out a protocol is file:.  I think this can be done independent of any other changes.
I'm hesitant to give up all information about which parts of the uri are mounting / hosting details and which are "real".  Although this came from a time when we had the same filesystems mounted under very different paths on different machines with much higher regularity, having the ability to "make the path relative" (for lack of a better term) seems generically useful (maybe we should have gone with a single integer rather than two parts of the path).  However, given the growing usage of containers and the trivial ability to be re-mount any level of path on the host to any level of path in the container it may make sense to lift all of this up to be a client configuration problem (which opens the door to much more complex re-writing rules).
"it has not been well used" is not a compelling argument to remove it (it is a compelling argument for better docs!), but "it does not actually solve the problem" is.

I'm also agnostic to the key name changes, maybe leaning slightly in the direction of pro, especially for mimetype and general reduction of the specialness of the documents.
I'll hold my hands up and say I hadn't noticed that stream_datum referenced descriptor. I agree that's not necessarily needed. If we move it to stream_resource do we need to keep the reference to run_start as well?
I agree with @tacaswell that the advent of containers is likely to make different path mounting more well-used. In fact I strongly encourage it because baked in assumptions of multiple machines using the same mount point has bitten me many times!
Also happy with the file:// and s3:// idea. If I were designing from scratch I might make it more structured
{
    "protocol": "file",
    "location": "/data/foo/bar"
}

But if :// is better for backwards compatibility then so be it :)

Exactly. The existing names aren't bad, but if we can use widely-known names and standards, developers don't even have to notice them.
Let's talk about the location of descriptor on a future call. I think we should keep run_start either way, to effectively save on a JOIN (a literal JOIN in SQL, a metaphorical JOIN in MongoDB...).
I like structure things, but if we care about covering s3: and others, we'd need to cover the full URL spec, as in:
{'scheme': 'file',
 'netloc': 'localhost',
 'path': '/a/b/c',
 'params': '',
 'query': '',
 'fragment': ''}

and I think the string form is probably the easiest thing to pass around in documents. Not a strongly-held opinion, though.

I'm not too attached to structure either but I don't think the full URL spec is a terrible thing to pass around. I think it might be best to see if anyone does hold strong opinions either way.

That is fair…and making me wonder whether Tiled should store the URI in a structured form at rest. That could enable indexing and more efficient filtering by scheme or netloc, which could come up…

From discussion:

Consider moving descriptor from stream_datum to stream_resource. This can be done by:

Updating schemas in event-model
Updating RE with a simple change: to tack the descriptor on to a partial stream_resource received from ophyd, and re-emit when it sees stream_datum pointing to a stream_resource in a stream that hasn't had one before.
No changes to ophyd at all

[For @tacaswell: Let the record show that @danielballan did not suggest this renaming.] Should we rename stream_resource and stream_datum because "stream" is overloaded? No good suggestions were made for a better name.

Even so, think it is the lesser evil. We can discuss. There is a tension here between what is easy for the producer and what is simpler (and performant) for the consumer.

I must confess to getting a bit lost among text descriptions of relational hierarchies, so I drew it out as a sanity check, does this look right?
Top is current state, bottom is proposed change:

Also tagging @coretl

If we move the descriptor to StreamResource that implies that we can have multiple StreamResource with overlapping keys for the same stream to support changing the configuration mid-stream. I suspect that is going to bring a bunch of complications down the line a well.

@tacaswell we already reduced the scope of StreamResource so that it only points to a single data_key, so I don't think keys could overlap? We also already need multiple StreamResource documents per Descriptor because we emit a new one per HDF file, and pausing/resuming is planned to start writing into a new file.
I don't think I have many opinions either way here, both options are the same complexity to produce in ophyd-async, and not much different in the run bundler, so happy with either.

Also tagging @DiamondJoseph for comment

Filling in the motivation for my terse notes above:
The Tiled adapter that loads Bluesky documents has to chase a lot of references, illustrated in Callum’s diagram, to answer the simple question, “What data do you have?” The overhead is especially unfortunate when it is asked to return pages of search results describing ~300 datasets at a time.
Placing descriptor and data_key in separate documents sets us up for more of this. It is maximally-normalized, but adds complexity and a JOIN. I don’t feel the trade-offs have been fully explored, but I think we should examine whether a little de-normalization (reissuing StreamResource for each Descriptor) could be worthwhile here.

Also also tagging @garryod

keeping Descriptor reference on StreamResource

Do you mean keeping the Descriptor reference on the StreamDatum, or moving it to the StreamResource? I think you mean the former, just checking.

We also already need multiple StreamResource documents per Descriptor

Fair, so in a given run we are going to have multiple StreamResource per key anyway.  Once we have that, I don't the the additional complication of "and they may point to different descriptors" adds much extra pain.
To be clear:

each run can have multiple streams (which are "concurrent")
each stream can have multiple descriptors (which are "serial")
each descriptor can multiple keys
each key can have multiple StreamResource
each StreamResource can have multiple StreamDatum

I think the following set of documents is valid (assuming only Stream* documents:
- Run (start document):
  - primary stream (synthetic concept)
    - descritptor 0 (document)
      - keyA (synthetic)
        - StreamResource0 keyA (document)
          - StreamDatum0 (document))
          - StreamDatum1 (document)
          - StreamDatum2 (document)
          - StreamDatum3 (document)
        - StreamResource2 keyA (document)
          - StreamDatum7 (document))
          - StreamDatum8 (document)
      - KeyB (synthetic)
        - StreamResource1 KeyB (document)
          - StreamDatum4 (document)
          - StreamDatum5 (document)
          - StreamDatum6 (document)
    - descritptor 1 (document)
      - keyA (synthetic)
        - StreamResource3 KeyA (document
          - StreamDatum9 (document))
          - StreamDatum10 (document)
      - KeyB (synthetic)
        - StreamResource4 KeyB (document
          - StreamDatum11 (document))
          - StreamDatum12 (document)
  - second stream (synthetic concept)
    - descritptor 2 (document)
      - KeyC (synthetic)
        - StreamResource5 keyC (document
          - StreamDatum13 (document))
          - StreamDatum14 (document)

That is we have two streams, one with A and B, one with C.  In the primary stream we have two descriptors (because we changed config or something) and with in the first descriptor we had to generate a second StreamResource.
I added levels of nesting for "stream" and "key" which are emergent / synthetic concepts....

Discussed with TC

TC is not unambiguous 😜

Maybe each TC should pick a UUID4 we can use.

keeping Descriptor reference on StreamResource

Do you mean keeping the Descriptor reference on the StreamDatum, or moving it to the StreamResource? I think you mean the former, just checking.

Yep, just completely said the opposite of what I meant. Edited the comment for posterity.

Discussed with TC
TC is not unambiguous 😜

Would you believe we had 3 Joes in the group before I joined Diamond? I've been explicitly Joseph for years, but it's become almost a necessity. Too used to people signing comments with just initials -JW.

@tacaswell I love the document hierarchy you produced as an example.

in the first descriptor we had to generate a second StreamResource.

Here is where I got lost. I haven't followed why there is more than one StreamResource per key, or what (typically) constitutes a StreamResource and what (typically) constitutes a StreamDatum.  From the above, it looks like a StreamDatum could include data from more than one "event"...in which case, what does a StreamResource represent?
I will gladly receive a link with a RTFM directive. :P    As someone who will certainly be a downstream consumer, I'm hoping to understand the proposed structure ahead of any decisions. :)

In descriptor0 the keys have an unequal number of StreamDatums. I gather this was intentional to reflect a realistic scenario. Is there a constraint that the total number of events for KeyA must equal the total number of events for KeyB?...and that they have matching sequence numbers? So in this example, StreamDatum{4,5,6} would presumably have more events than most StreamDatums under KeyA.

It is enforced by the run bundler that each key must have the same number of events and they have matching sequence numbers. It is also currently enforced by the run bundler that each StreamDatum for each key must be the same length as those for the other keys in the same descriptor. This is an implementation detail, but could be made a guarantee if helpful...

Thanks, @coretl !  I think having it as an expectation (rather than a guarantee) is sufficient, and handled by the RunBundler makes sense.
So, I would then say that the details under descriptor 0 are technically valid, but should never be present in practice.  There should be the same number of StreamDatums under Key A and Key B.

A proposal developed with @callumforrester today:
Instead of specifying a file x/y/z mounted at /a/b/ as:
{
  "root": "/a/b/"
  "resource_path": "x/y/z"
}

how about
{
  "uri": {
    "scheme": "file",
    "path": ["a", "b", "x", "y", "z"],
    "netloc": "localhost"
    "query_params": [],
  }
  "facets": [
    {"segments": {"start": 0, "end": 2}, "feature": "root"}
  ]
}

That is, the URI is given as an object, with a side band ("facets") explaining the semantics of certain segments of the path. This enables:

A simple, self-contained URI with no explanation required for devs
A path to non-file-based storage (e.g. s3 instead of file)
A defined (perhaps required?) place to specify the root
The possibility of saying more than just root, adding other optional facet types

I learned about a "facets" comment from a UI dev working for the other Bluesky, explaining why Bluesky posts don't just use Markdown. The requirements resonate with ours. A developer can freely ignore the facets if they don't (and don't want to) understand them, but they are can be used to give extra meaning to the path.

For completeness I should mention that @tacaswell originally considered something like:
{"resource_path": "/a/b/x/y/z",
 "separator": 2
}

which, similarly uses a side-band. The "facets" proposal above is more flexible in that it can describe any segment and more than one.

Coming late to the party here, but I've got a few thoughts from a consumer's perspective...
Using URIs to specify resource locations is ideal but they should be encoded as described in IETF RFC 3986 (i.e. a string) - encoding them as any kind of struct will require users to write custom deserialization code when Url::from_string exists in all popular languages.
Absolute file paths are impossible where shared file systems are concerned as the file system can - and will be - be mounted at an arbitrary path. As such it only makes sense to use relative paths when describing the location of files and have the base directory agreed out of band - or perhaps with an additional field dedicated to this purpose though this has us defining a higher level protocol for resolving file systems.

I like that argument for URI as string.

I'm really glad you chimed in @garryod. Musing on:

Absolute file paths are impossible where shared file systems are concerned as the file system can - and will be - be mounted at an arbitrary path.

...that is true at NSLS-II as well (and has been a source of many headaches).
From RFC 8089, the "file" URI scheme:

The "host" is the fully qualified domain name of the system on which
the file is accessible.  This allows a client on another system to
know that it cannot access the file system, or perhaps that it needs
to use some other local mechanism to access the file.

with examples including file://host.example.com/path/to/file
I wonder if we can go with URIs like:
file://data1.nsls2.bnl.gov/chx/proposals/2024-1/pass-333333/assets/image00001.tiff
file://nfs1.diamond.ac.uk/i22/data/2023/cm33873-5/i22-723667-waxs-hdf.h5
file://localhost/a/b/c.h5  # equivalent to the shorthand, file:///a/b/c.h5

For testing, dev, and simple deployments with local storage, we can just use localhost with the local absolute path. For facility-scale deployments, we can, as @garryod said,

use relative paths when describing the location of files and have the base directory agreed out of band

The out of band mechanism could, for example, be local configuration mapping the domain in the URI to the local mount point, e.g.
mounts:
- domain: data1.nsls2.bnl.gov
  root: /nsls2/data1

I like that this would use existing standards, a single string URI, and no additional fields in the document that would require explanation/documentation.

From RFC 8089, the "file" URI scheme:

The "host" is the fully qualified domain name of the system on which
the file is accessible.  This allows a client on another system to
know that it cannot access the file system, or perhaps that it needs
to use some other local mechanism to access the file.

Hadn't come across this before, but in the absence of any better suggestion it seems like the way to go even if it's only currently a Proposed Standard.

The out of band mechanism could, for example, be local configuration mapping the domain in the URI to the local mount point, e.g.
mounts:
- domain: data1.nsls2.bnl.gov
  root: /nsls2/data1

Feels like there should be an of the shelve way of doing this - like the /etc/hosts file for IPs and domain names - but a quick google turned up nothing. Has anyone come across something that would fulfil this role?

...that is true at NSLS-II as well (and has been a source of many headaches).

This is exactly the reason why we have root in the resource/datum and root_map on databroker to begin with.

I wonder if we can go with URIs like:

I hope we can do that re-writing at the "I'm consuming documents and pushing them into tiled" stage and not at the "I generated these in ophyd" stage as I do not think we want to inject the details of mounting into the ophyd devices (as reflecting on how root/root_map has worked at NSLS-II the main problem is we lost control of the root being set correctly (because it was done at the ophyd layer on each device in each profile)).

The root + resource_path formulation effectively uses the local mount point (e.g. /GPFS) to identify a given networked storage volume. This is potentially ambiguous. In the bad old days, there were examples on the floor of the same mount point on two different hosts pointing to two different storage clusters. There was no valid root_map solution to that, and we had to write some custom spaghetti to deal with certain situation.
Using the hostname of the storage array is unambiguous.
When possible, I think it would be best to set the URI from the start in ophyd, so that streaming consumers that happen to be on different hosts (and may have a different view of the storage) can work with it. If the engineer configuring ophyd just hands us an absolute path, we can emit file://localhost/{path}. A consumer could potentially flag that, or even rewrite based on examining the {path].

Offline, @tacaswell argue that if ophyd puts the storage host in the URI, and then storage changes, we have to update all the "clients" (ophyd config). I think he's convinced me that ophyd should just emit localhost URIs and that some downstream consumer, centrally managed, should rewrite them.
But there is room for variability here: this is just an argument at the level of what to recommend.

I think he's convinced me that ophyd should just emit localhost URIs and that some downstream consumer, centrally managed, should rewrite them.

I may be missing something here, but how will the downstream consumer (tiled?) know what to rewrite them to without knowledge of how filesystems are mounted on each and every machine a ophyd device runs on? Or is the thought that this is less ownerous than having each ophyd device configured with this?

I'm enthusiastic about removing path_semantics, renaming spec and resource_kwargs. I'm on board with renaming root (our devices should always be mounting either /device/data = /localhost/dls/ixx/data or /device/dls/ixx/data = /localhost/dls/ixx/data, so we avoid your bad old days by having our own different bad old days, which we're still in).
I'm opposed to renaming stream_resource and stream_datum: I think it's clear (to me) what they mean and how they relate to existing documents, without being easily confused with them.

Help me to avoid losing focus here.  localhost refers to the workstation running the event-model process?  Not the IOC that writes the image?
I'm thinking of area detectors with their own filesystem that are not mounted on the host where the event-model process is running.

Or is the thought that this is less ownerous than having each ophyd device configured with this?

I think the actual problem with the current scheme is that we left it up to the ohpyd classes where to do the root / resource path split (and to do the windows <-> posix remapping) and in general it was in the wrong place so if we move to using a uri but don't change where it is configured I do not think we will do anything other than change the spelling (if you use root_map to convert root -> hostname and tack on file:// you can get a uri).
If we continue let ophyd/RE hold the mapping (but spell it with in-band encoding) then the following story will happen:

ophyd knows that /data/facility/ is from data1.facility.gov on the server running det1
we buy data2, do not copy the files from data1 to it and change the mount in the IOC server to be data2 instead of data1 (because the IOC does not need to be able to see old data)
we now have to track down every single place that we might have an ophyd object that looks at or we happily emit very wrong documents

Not exactly the same problems we have now, but I think they will be terrible in about the same ways.
Hence, I think the mapping from  file://localhost/absolute/path to file://data.host.dns.entry/relative/path either needs to be done "at the edge" by the code that writes these documents down to persistent storage (as if everything is streaming I think it is reasonable to expect that either you have consistently mounted your network storage or the client who has storage mounted differently know how to do the transformation) so that we only have to maintain a shadow copy of the IOC fstab(s) in one place or it the AD IOCs need to provide a PV that provides the root -> host mapping generated by reading fstab (or the file shuffling service or what ever).  If we are being really ambitious a PV that gives the uri directly (rather than the absolute filename from the point of view of the IOC) that would be even better.
We need to make sure that where ever we land on this we don't make the low-infrastructure use case (dev, testing, small deployments) does not get needlessly complicated.

@prjemian Good point of clarification! I think we can stipulate that if ophyd hands us a localhost URI, it means localhost from the point of view of the process that is writing the file (i.e. the IOC).
It's yet to be settled in the discussion where/how we should recommend that this gets mapped into a portable representation (file://data.host.dns.entry/relative/path).

@tacaswell This is exactly why we can never buy data2 in our current scheme (GDA et al.)

How about ophyd produces URIs like file://ioc-hostname/absolute/path? That is knowable from CA/PVA, and would at least tell you who wrote the file at that location...

file://ioc-hostname/absolute/path seems like a good middle ground to me.  No extra PVs and a service that persists the documents can own the IOC -> mount path (so we only have 1 shadow copy of fstab) can reliably fix it with no heuristics / looking at values in other documents.  Down side is that it assumes your IOCs have dns names... are uris like file://ip.ad.res.ss/absolute/path allowed? Or should we just not take the case of "I am running a big facility but don't use DNS" seriously?
My understanding of localhost in this context is "YOLO, hope you have consistent mounts 🙃 " like passing around a "bare" /absolute/path and

All our IOCs are in DNS, but the vxWorks ones don't know anything about it... So we can get IP address out of cainfo, reverse lookup the name in DNS, then put that in the URI. Probably not the end of the world if the entry is not in DNS and IP addresses get into the URI though?

Yes, an IP address in the URI is fine.


add xcit

Thank you for your contribution to the MindCV repo.
Before submitting this PR, please make sure:

[ ] You have read the Contributing Guidelines on pull requests
[ ] Your code builds clean without any errors or warnings
[ ] You are using approved terminology
[ ] You have added unit tests

Motivation
(Write your motivation for proposed changes here.)
Test Plan
(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)
Related Issues and PRs
(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)

目前不需要中文readme，需要删掉中文readme


Added Docker Hub exporter to list of third-party exporters

Adding details of a simple Docker Hub exporter for tracking statistics about pulls/stars etc

Please have a look at https://prometheus.io/docs/practices/naming/#metric-names. Counters should end on _total and not _count.
Also have a look at https://prometheus.io/docs/instrumenting/writing_exporters/ including the section about port numbers.

Both very valid points!
I've pushed an update to the repo to resolve the counter issue and to move the tracking of stars to be a gauge from a counter. With regards to the port number, i've updated it to a new port and listed that on the wiki.
Let me know if there is anything else i've missed.

@Rucknar Having the stars as a gauge is correct, but then please also omit the _total suffix (we only use that on counters, to distinguish them from gauges). Also, the convention is to use the plural of whatever you're counting / measuring:

docker_hub_pulls_total
docker_hub_stars

(you may also want to update the sample metrics section in the README.md)

Merging this for now...

Thanks @juliusv
I've made the recommended adjustments above and pushed.


The job log hint in the job listing interface could be better

Expected Behavior
The current hint for logs in the job listing reads To see the logs of a job, use: 'nerd job logs <JOB-NAME>' but this might confuse people to think something went wrong. If we change it to something like All you jobs are listed below, to see logs use: 'nerd job logs' . This refers the use to the listing immediately

I updated the hint in #369, but I would be glad to get your opinion about it @advanderveer
https://github.com/nerdalize/nerd/pull/369/files#diff-96d999c24a92f857fd7c67b513d7a881R66


Tagging volumes snapshots based on their previous tags...

Idea from @charterresources from #1

Did some brief investigations into this... would take some effort, will need to...

[ ] Sleep after creating AMI(s) and wait for all snapshots to appear
[ ] Because snapshots are not directly associated with an AMI, only a volume (with a description content which mentions the AMI) would need to scan through the snapshots to find the right one
[ ] Could then tag every snapshot with whatever tags that volume had

If only a few AMIs are created this is possible, but for a larger infrastructure/deployment this will fail, because when you schedule more than a few AMIs, they don't all run at once they can schedule to actually run a little later, which would push us over 5 minutes in Lambda if we had to wait that long.
Some ideas to achieve this might be to on a "second" run of this to try to tag older snapshots, as it's creating new AMIs and snapshots.  Or to create a second lambda which does this automatically, but this feels like over-complicating something I want to keep fundamentally simple.  Input/ideas welcome.

If only a few AMIs are created this is possible, but for a larger infrastructure/deployment this will fail, because when you schedule more than a few AMIs, they don't all run at once they can schedule to actually run a little later, which would push us over 5 minutes in Lambda if we had to wait that long.

@AndrewFarley Are you sure about this? Looking at my 19 AMIs, their time of creation are within 23 seconds of start time of lambda function.
~Samet

@charterresources I've seen this happen in the past numerous times (been working on AWS for way too long).  So, under good conditions this never happens.  I'm okay with implementing it, but it won't be enabled by default because I have no way to fail gracefully if I go over the 5 minute timeout of Lambda unless I implement state management and a DLQ and a retry mechanism based on the DLQ.  But, I still plan to implement it because those snapshots without tags are really annoying.  It's just a fundamental flaw in the way AWS does AMI snapshots (without tags) that I have to try to compensate for.

@charterresources If you want to test what I'm talking about... try running your lambda 5 times in a row very quickly, so you have 100 AMI's queue'd up, and then go look at the snapshot creation times, not the AMI creation times.  In some brief investigation tests earlier this week I saw the problem on one of my environments... the create AMI request went through, but then for about 10 minutes I could not query about that AMI, the API responded with "unknown AMI".  This is what I'm talking about, unfortunately.  They have an "eventually consistent" model we have to work around.

@AndrewFarley
Great!!
thank you Andrew...


Auto formatting doesn't use editor preferences.

This issue was originally filed by @giry

What steps will reproduce the problem?

Go to tools->preferences->editor
Set "4" in "displayed tab width".
Open file with a valid dart code.
Use "format" option from context menu.

What is the expected output? What do you see instead?
The formated code has 2-space width indentation, but should use 4-space.
What version of the product are you using? On what operating system?
Dart Editor 1.2 (build 33014)
Please provide any additional information below.

This issue has been moved to dart-lang/dart_style#328.


Cannot run element explorer from command line on windows. Keep getting Error: Invalid character code:800A03f6. Setting i have in elelemtexlporer is #!/usr/bin/env node. Please suggest how to set this setting for windows.

Hi there!
Cannot run element explorer from command line on windows. Keep getting Error: Invalid character code:800A03f6. Setting i have in elelemtexlporer is #!/usr/bin/env node. Please suggest how to set this setting for windows.

Hi there,
Please follow the Issue Template when opening new issues. Otherwise it's unclear what the issue is and how we can actually help.
I'm going to close this issue, please re-open a new one following the issue template linked above. Thanks! 😄

Can you please help us with information. I had went to the website and put in the request.
From: Nick Tomlin<EMAIL_ADDRESS>Sent: Thursday, September 22, 2016 12:52 PM
To: angular/protractor<EMAIL_ADDRESS>Cc: Venkat Pandiri<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [angular/protractor] Cannot run element explorer from command line on windows. Keep getting Error: Invalid character code:800A03f6. Setting i have in elelemtexlporer is #!/usr/bin/env node. Please suggest how to set this setting for windows. (#3593)
Closed #3593https://github.com/angular/protractor/issues/3593.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/angular/protractor/issues/3593#event-799166149, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVJ2lEQh3wxTpCmDLIca015dhJvbbWaMks5qsrI-gaJpZM4KEGWl.
This e-mail and any attachments, contain Genuine Parts Company confidential information that is proprietary, privileged, and protected by applicable laws. If you have received this message in error and are not the intended recipient, you should not retain, distribute, disclose or use any of this information and you should destroy this e-mail, any attachments or copies therein forthwith. Please notify the sender immediately by e-mail if you have received this e-mail in error.

The issue queue is not for support requests (please read the issue template or the CONTRIBUTING.md)
From the the getting help section of the README:

Please ask usage and debugging questions on StackOverflow (use the "protractor" tag) or in the Angular discussion group. (Please do not ask support questions here on Github.)

Yes  I was asking on the stacktrace not sure. Where the missing link is
From: Nick Tomlin<EMAIL_ADDRESS>Sent: Thursday, September 22, 2016 1:22 PM
To: angular/protractor<EMAIL_ADDRESS>Cc: Venkat Pandiri<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [angular/protractor] Cannot run element explorer from command line on windows. Keep getting Error: Invalid character code:800A03f6. Setting i have in elelemtexlporer is #!/usr/bin/env node. Please suggest how to set this setting for windows. (#3593)
The issue queue is not for support requests (please read the issue template or the CONTRIBUTING.md)
From the the getting helphttps://github.com/angular/protractor#getting-help section of the README:
Please ask usage and debugging questions on StackOverflow (use the "protractor" tag) or in the Angular discussion group. (Please do not ask support questions here on Github.)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/angular/protractor/issues/3593#issuecomment-248969578, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVJ2lHn-SrjtYzwAXjj6GsB3DSBvOWQHks5qsrlTgaJpZM4KEGWl.
This e-mail and any attachments, contain Genuine Parts Company confidential information that is proprietary, privileged, and protected by applicable laws. If you have received this message in error and are not the intended recipient, you should not retain, distribute, disclose or use any of this information and you should destroy this e-mail, any attachments or copies therein forthwith. Please notify the sender immediately by e-mail if you have received this e-mail in error.

Look at the image below
<EMAIL_ADDRESS>From: Nick Tomlin<EMAIL_ADDRESS>Sent: Thursday, September 22, 2016 1:22 PM
To: angular/protractor<EMAIL_ADDRESS>Cc: Venkat Pandiri<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [angular/protractor] Cannot run element explorer from command line on windows. Keep getting Error: Invalid character code:800A03f6. Setting i have in elelemtexlporer is #!/usr/bin/env node. Please suggest how to set this setting for windows. (#3593)
The issue queue is not for support requests (please read the issue template or the CONTRIBUTING.md)
From the the getting helphttps://github.com/angular/protractor#getting-help section of the README:
Please ask usage and debugging questions on StackOverflow (use the "protractor" tag) or in the Angular discussion group. (Please do not ask support questions here on Github.)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/angular/protractor/issues/3593#issuecomment-248969578, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVJ2lHn-SrjtYzwAXjj6GsB3DSBvOWQHks5qsrlTgaJpZM4KEGWl.
This e-mail and any attachments, contain Genuine Parts Company confidential information that is proprietary, privileged, and protected by applicable laws. If you have received this message in error and are not the intended recipient, you should not retain, distribute, disclose or use any of this information and you should destroy this e-mail, any attachments or copies therein forthwith. Please notify the sender immediately by e-mail if you have received this e-mail in error.

I think there's a communication breakdown here. Please direct your questions to StackOverflow or Gitter.
I'm going to lock this thread to avoid any more confusion. Thanks! 😄


Hartree fock

This should create working multi electron HF.
TODO

[ ] Check that multipliers are fine with Coulomb and Exchange operators
[ ] Check results
[ ] Add call to total energy

There are still some problems here.
SCF has energy increasing instead of going down.
This probably means that fock matrix has some issues.


[New Feature]: Music ON OFF button in home page

🎮 feature Request
feature logic and basic description:
There is a issue in home page once after opening the website we are not geeting music but if we press button in the home page then only music is getting started
Point down the features
feature points:
-Now the user can on or off Music in home  page
Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

@ayush-t02 Please assign me the issue


Fix Changes Job in CI

the quotes ('') in the changes job failed to glob match the changed files, and thus the check job was not run. This PR fixes that, as well as add seccomp directory in changes.

Codecov Report

Merging #306 (a17e4f2) into main (74ed630) will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #306      +/-   ##
==========================================
- Coverage   77.19%   77.17%   -0.02%
==========================================
  Files          43       43
  Lines        6139     6139
==========================================
- Hits         4739     4738       -1
- Misses       1400     1401       +1

@utam0k Please take a look, I have changed the glob pattern


Check for Log4j CVE-2021-44228 attacks

A proposal for a DataScript which detect CVE-2021-44228 attacks in the HTTP request header

Script is not yet tested in detail, waiting for test results.

Not sure without profiling it, but would a single Lua pattern match call be more effecient than multiple call-outs to string.contains()?

@thebertster I agree, will check and fix it. We can probably but both checks - for the attack and for the evasion of the attack into a single regex.


skip standalone sequences

Sequence created without table should not be dropped from dump

@xronos-i-am Is this still desired functionality? Do you have a scenario where a sequence is created outside of a table, where it shouldn't be dumped?
I could see a scenario where someone creates a sequence owned by some column, and we'd want to dump it.
If you could explain the use case more, that would help. Thanks!

Hi. Yes it is still desireable. We use sequences without tables sometimes. When we assume that a related table may appear in the future, but we don't need it for now. We use sequence as autoincremented counter in that case


🐛 actions property in NavigationAppBar not showing the widget in fluent_ui: ^4.0.0-pre.2

In previous version the WindowCaption was showed but to this prerelease the window buttons were not showing.

Sorry, but I can not reproduce it :/
Can you try the following steps?

close the app
run flutter clean
run the app again

I already did that. The widgets in actions property will show only if the automaticallyImplyLeading set to true. This bug occur in version 4.0.0pre2

Sorry, but I can not reproduce this issue in the example app.
automaticallyImplyLeading: true:

automaticallyImplyLeading: false:


feat: add per host feature

Which issue(s) this PR fixes

Fixes #307
Additional comments

https://github.com/TOSIT-IO/tdp-collection/pull/789 is needed to remove playbooks with import_playbook.
Agreements

[x] I hereby declare this contribution to be licensed under the Apache License Version 2.0, January 2004.
[x] I hereby agree to grant TOSIT a copyright license to use my contributions.

I think there's a problem when the plan is modified : the 'host' column is not refreshed when i execute several commands
eg. commands and results below :
tdp plan dag :
Operations:
  deployment_id    operation_order  operation                                 host    start_time    end_time    state    logs
---------------  -----------------  ----------------------------------------  ------  ------------  ----------  -------  ------
              1                  1  exporter_jmx_install                      None    None          None        PLANNED

tdp plan run --hosts mdrutel-edge-01 knox_ranger_install :
Operations:
  deployment_id    operation_order  operation            host             start_time    end_time    state    logs
---------------  -----------------  -------------------  ---------------  ------------  ----------  -------  ------
              1                  1  knox_ranger_install  mdrutel-edge-01  None          None        PLANNED

tdp plan dag :
Operations:
  deployment_id    operation_order  operation                                 host             start_time    end_time    state    logs
---------------  -----------------  ----------------------------------------  ---------------  ------------  ----------  -------  ------
              1                  1  exporter_jmx_install                      mdrutel-edge-01  None          None        PLANNED

This is because, when a plan is performed when a plan is already in the database, the new plan is merged with the old one. I put host=None in from_dag and from_failed_deployment to fix that.
diff --git a/tdp/core/models/deployment_log.py b/tdp/core/models/deployment_log.py
index 873e55f..76bc8c4 100644
--- a/tdp/core/models/deployment_log.py
+++ b/tdp/core/models/deployment_log.py
@@ -178,6 +178,7 @@ class DeploymentLog(Base):
             OperationLog(
                 operation=operation.name,
                 operation_order=i,
+                host=None,
                 state=OperationStateEnum.PLANNED,
             )
             for i, operation in enumerate(operations, 1)
@@ -326,7 +326,10 @@ class DeploymentLog(Base):
         )
         deployment_log.operations = [
             OperationLog(
-                operation=operation, operation_order=i, host=None, state=OperationStateEnum.PLANNED
+                operation=operation,
+                operation_order=i,
+                host=None,
+                state=OperationStateEnum.PLANNED,
             )
             for i, operation in enumerate(operations_names_to_resume, 1)
         ]

I also discover a bug with resume, now host is correctly read when running tdp plan resume. I added a test for checking that.

Hi,
How can we declare multiple hosts for an operation ?
tdp plan run --hosts host1,host2 operation doesn't seem to work

I dont' know why this does not work, I use the standard multiple of click but it looks like you have to specify the option as many times as there are hosts (--hosts host1 --hosts host2)

Yep, it's not possible to have an option that takes an undefined number of arguments with click. I suggest that we either:

Implement it by hand, by parsing a --hosts host1,host2,host3 string on ,.
Keep the click default behavior but remove the s to the --host option.

I dont' know why this does not work, I use the standard multiple of click but it looks like you have to specify the option as many times as there are hosts (--hosts host1 --hosts host2)
Actually, it's bound to the comment made by Paul :p

We can add multiple hosts with --hosts
We can only specify 1 host with --hosts => Should we rename the option with -- host ?

I don't know if this issue is the right place, but it will be nice if tdp nodes could also give the hosts on which the operation could be launched ^^'

I don't know if this issue is the right place, but it will be nice if tdp nodes could also give the hosts on which the operation could be launched ^^'

Done.

I don't know if this issue is the right place, but it will be nice if tdp nodes could also give the hosts on which the operation could be launched ^^'

Done.

Thanks a lot ^^


Conflict with Hide Servers / Channels Plugin

First of all, thanks for your hard work. This theme looks great!
As the title suggests, this isn't a bug with the theme per se, but I hope you could point me in the right direction to solve the conflict with this plugin.
The plugin provides buttons in the UI and keybindings to collapse the channel list / server list. Both buttons appear correctly with and without Slate, and the button/keybinding to hide the servers works with and without Slate as expected. However, the button/keybinding to hide the channel list causes the channel list to briefly collapse and then reopen (the plugin does work without Slate).
I've looked at the code for both Slate and the plugin in question, but my CSS/JS-Fu is essentially nonexistent and I wasn't able to make them play nice together. I decided to open this issue in the hope that you could fix the conflict or at least help me figure out how to fix it myself.
Thanks!

Fixed.


maximum call stack size at friends.js line 1137

Describe the bug
maximum call stack size
Versions
v4.19.3
Screenshots and Error Logs

I haven't been able to reproduce this.

this seens to happens when the logged in account has a lot of friends, and this makes the event 'friendsList' not to be triggered, causing the property 'myFriends' to be empty

actualy the friendsList event doesnt seen to be triggered at all, not sure why,

Are you saying that friendsList is never triggered, or that it's not triggered when you get the error? Or do you get the error every time you log on?

friendsList isn't being triggered, but im not sure anymore whats causing it, at first i thought it was the maximum call stack error, but on my tests it wasn't being triggered even without call stack error, but the weird thing is, at first log on, it was triggering, but on next ones, it wasn't, and i've checked the myFriends property, it was empty making the script unable to manage friend list
so, something is making the friendsList event unable to be triggered(also making the property myFriends empty), but the script seemed to be fetching users details(user event was being triggered)..
do you have any tests in mind so i can help to fix that issue?
to be more specific:

the account used does not have any sort of limitations in steam, and works just fine, no trade holds, bans, or whatever limitations steam have but market usage, the account does not use the steam market.
more users seems to be experiencing this issue.
a specific user is having the maximum call stack error, and after remove some friends, the error disappears.
its possible that more users is having the maximum call stack error, since only one have reported it, but others have similar issue.

If I could get debug output from you, that would be helpful.
Client.on('debug', (msg) => {
   console.log(msg); // or whatever custom debug log function you want to use
});

this Got incomplete message; expecting 4864 more bytes seems to be the problem

if anyone else is having this issue, a failover to https://developer.valvesoftware.com/wiki/Steam_Web_API#GetFriendList_.28v0001.29 its a good idea.

this Got incomplete message; expecting 4864 more bytes doesn't sound harmless

It is, that's normal.
I really don't know why you wouldn't be receiving your friends list. It gets pushed down automatically by the CM; it's not something that can be requested. The same is true of friend persona states. I would expect to see a ClientFriendsList handled message line, but it just isn't there. Are you sure that's the entire debug output? There should be some more output around when the client connects and logs on.

this Got incomplete message; expecting 4864 more bytes doesn't sound harmless

It is, that's normal.
I really don't know why you wouldn't be receiving your friends list. It gets pushed down automatically by the CM; it's not something that can be requested. The same is true of friend persona states. I would expect to see a ClientFriendsList handled message line, but it just isn't there. Are you sure that's the entire debug output? There should be some more output around when the client connects and logs on.

yeah, that's everything. some accounts just doesn't download the friend list somehow, and i have no idea why

I get this error sporadically too, but on a different line:
Exception in PromiseRejectCallback:
steam-user/components/friends.js:1190
			return resolve(user);


Restrict tl.dot size >= 16x16x16 only for CUDA

The core Triton is a small number of people, and we receive many PRs (thank
you!).  To help us review your code more quickly, if you are a new
contributor (less than 3 PRs merged) we ask that you complete the following
tasks and include the filled-out checklist in your PR description.
Complete the following tasks before sending your PR, and replace [ ] with
[x] to indicate you have done them.

[x] I am not making a trivial change, such as fixing a typo in a comment.

[x] I have written a PR description following these
rules.

[ ] I have run pre-commit run --from-ref origin/main --to-ref HEAD.

Select one of the following.

[ ] I have added tests.

/test for lit tests
/unittest for C++ tests
/python/test for end-to-end tests

[x] This PR does not need a test because It's obvious.

Select one of the following.

[x] I have not added any lit tests.
[ ] The lit tests I have added follow these best practices,
including the "tests should be minimal" section. (Usually running Python code
and using the instructions it generates is not minimal.)

@antiagainst, we had a discussion about that and one of the possible conclusion was to make this restriction backend specific? Did you have plans to work on this?

once this PR lands you can adjust the restriction downstream:
https://github.com/triton-lang/triton/pull/3908


feat: Update to build against Angular 15

Update to build against Angular 15

It seems the tests found in angular-reactive-validation/src/**/*.spec.ts no longer run. Could you look into this?
In general I think the project structure is no longer matching the recommended Angular structure at this time. I'll be sure to fix that before the Angular v16 release (and also set up proper GitHub actions), but for now I hope we can still "duct tape" this issue.

In general I think the project structure is no longer matching the recommended Angular structure at this time. I'll be sure to fix that before the Angular v16 release (and also set up proper GitHub actions), but for now I hope we can still "duct tape" this issue.

I'm working on that as we speak. It doesn't seem to be much work. Hopefully done soon.

The restructuring is done. I'll work on automating build, test, and release some other time. Thanks for your effort 💪.


how to make a list of the file names to use for training/testing?

ls data/.wav | tail -n+10 > train_files.txt
ls data/.wav | head -n10 > test_files.txt
I don't know "| tail -n+10" mean what?

bash command


Crushing wheels controller causing blocks to be unmovable by mechanical piston (sticky & regular)

Describe the Bug
I was working out a contraption design, and when testing if it would move correctly, I encountered a strange bug where the sticky/regular mechanical piston would give an "Unable to assemble contraption" message, citing an unmovable block on the contraption.  These blocks should all have been movable, however.  The ones I saw with this design included encased fans, crushing wheels, and shafts.  Which block was considered unmovable seemed to depend on which direction the piston was pushing the contraption.  When I tried breaking the cited block, it allowed me to extend the mechanical sticky piston like normal, but when trying to retract the mechanical sticky piston, it then cited an entirely different unmovable block.  I've attached a .zip file with the schematic I was using (which I tested in multiple singleplayer worlds to confirm the issue was consistent).  I have no clue what would be causing this, any help is appreciated.
schematic.zip
Reproduction Steps

Download & unzip the schematic.zip file, then place the NBT file in the schematics folder.
Open a world and paste the schematic.
Using a mechanical piston or mechanical sticky piston and a handcrank, attempt to push the contraption from any side.
For me, it always gives an unmovable block error preventing the contraption from assembling, with a different block being unmovable depending on the direction of the mechanical piston.

Expected Result
None of the blocks on the contraption should be unmovable, so one would expect it to move when pushed or pulled by a mechanical piston.
Screenshots and Videos
No response
Crash Report or Log
No response
Operating System
Windows 11
Mod Version
0.5.1f
Minecraft Version
1.20.1
Other Mods
Create Enchantment Industry 1.2.16
Create (Fabric) 0.5.1.d, build 1417
Fabric API 0.92.1
(Contraption was built with blocks from Extended Drawers 2.1.1, but when testing I forgot and removed this mod by accident.  However, the bug still occurred without Extended Drawers installed.)
Additional Context
No response

Okay, so I've done a bit more testing with this, and it seems to be an issue with the crushing wheels controller (I think that's what it was called—it's the phantom block that exists between two crushing wheels). Anytime you try to push/pull a block in a path that goes through this phantom block between two crushing wheels, it registers whichever block would go through the space where the controller is as "unmovable".

hmmmm, dont know if that is supposed to happen. Maybe rename the issue with the discovery to something like "Contraption unmovable when going through crushing wheel controller."

Also occurs on forge, an issue needs to be made there


fix: pg-meta v0.84.1

https://github.com/supabase/postgres-meta/compare/v0.83.2...v0.84.1

Pull Request Test Coverage Report for Build<PHONE_NUMBER>3
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.03%) to 59.748%

Files with Coverage Reduction
New Missed Lines
%

internal/gen/keys/keys.go
5
12.9%

Totals

Change from base Build<PHONE_NUMBER>8:
-0.03%

Covered Lines:
6356

Relevant Lines:
10638

💛  - Coveralls


TrifleJS Support?

Has anyone checked for compatability with triflejs?  Even though we would have to spin up a windows machine to run it, it would be better than manually doing regression tests in IE.

Closing this for now. If someone has written a plugin feel free to report here


Incorrect Use of Terms in Calorie Counter Project

Describe the Issue
We got an email in the support inbox recently about the incorrect use of the terms "surplus" and "deficit" in the calorie counter project.

Currently the calorie counter project uses these terms the opposite way. It shows "x Calorie Surplus" when the number of calories remaining is less than the daily calorie budget, and "x Calorie Deficit" the number of calories consumed goes over the daily budget.
I think we have a couple of options here:

Update the description and seeds starting from step 77 (currently id 63c9e55b4b06c11fff555c64) to the final step to have learners write a ternary operator that returns Surplus if remainingCalories is greater than or equal to 0, and return Deficit if not (const surplusOrDeficit = remainingCalories <= 0 ? 'Surplus' : 'Deficit';).
Since the terms Surplus and Deficit can still be confusing, update the project to use the text "X Calories Remaining" / "X Calories Over" or something similar, and for the text color to be green if remainingCalories is greater than or equal to 0, and red if not. The variable name surplusOrDeficit could still be used, but it may be better to update it to something more general like overOrUnder. If we do update this variable, we should also update the classes in the HTML and CSS from surplus and deficit to over and under, too.

It would sound nicer to use "X Calories Remaining" / "-X Calories Remaining", but I believe this project is the only one to teach the Math.abs() method at the moment, so wording like "X Calories Remaining" / "X Calories Over" would allow us to keep teaching it here.
Or if someone has another suggestion for a way to improve this project, feel free to share your thoughts and we can discuss it.
Affected Page
https://www.freecodecamp.org/learn/javascript-algorithms-and-data-structures-v8/learn-form-validation-by-building-a-calorie-counter/step-77

I vote for option 1.
I just think we need to add a short explanation of what a deficit and surplus is if we haven't done so already.
But I think the changes proposed in option 1 should work.

Can i take on this issue?

@abdqudus
This issue has not been marked as ready for contribution.
Issue marked with help wanted or first timers only are available for contribution.

hey folks, just to clarify as i am working on this, we are updating the wording on step 77 to the following  and every step after that to the end :
"....1st para
Use a ternary operator to assign the variable surplusOrDeficit the value "Surplus" if remainingCalories is greater than or equal to 0, and "Deficit" otherwise "
@scissorsneedfoodtoo also const surplusOrDeficit = remainingCalories <= 0 ? 'Surplus' : 'Deficit'; this shows remainingCalories<= 0 , taks requirement is remainingCalories >= 0
would appreciate you feedback before raising a PR

Thanks for your input, @jdwilkin4  and @naomi-lgbt. And thanks for your interest on working on this, @RGHANILOO. I did a bit more research and updated my comment above.
Just to clarify, here's what I think we should do:

On step 77, give a brief explanation of what a caloric surplus and deficit are, and modify the instructions so learners write a ternary string that returns Surplus if remainingCalories is less than 0, and returns Deficit if not (const surplusOrDeficit = remainingCalories < 0 ? 'Surplus' : 'Deficit';).
The text could be something like this:
You need to know if the user is in a caloric surplus or deficit.  Burning as many calories as you consume is called maintenance, and can be thought of as a surplus or deficit of `0`, depending on your goals.

Declare a `surplusOrDeficit` variable. Then use a ternary operator to set `surplusOrDeficit` to the string `Surplus` or `Deficit` depending on whether `remainingCalories` is less than `0`. If it is less than `0`, then `surplusOrDeficit` should be `Surplus`. Otherwise, it should be `Deficit`.

The seeds in the remaining steps 78-94 and the solution in step 94 should use the updated ternary operator const surplusOrDeficit = remainingCalories < 0 ? 'Surplus' : 'Deficit';. You could do a search and replace to modify all the seeds / solution for the English version of the project files.

The CSS variables for .surplus and .deficit need to be swapped, so .surplus uses var(--light-pink) and .deficit uses var(--light-green). You could do a search and replace to modify all the seeds / solution for the English version of the project files.

Hey @scissorsneedfoodtoo ,
Thank you for your detailed response, i will start working on this now.


Split of Arxiv

Hi! Thanks for sharing the elegant codebase. I have a problem regarding the dataset split of the ogbn-arxiv. I notice that the split is given by the ArxivSplitter, which is a 10-fold split, and 80% of the data are adopted as the training set, which is different from the original split. However, it seems the performance of GCN in Table 3 is taken from OGB's leaderboard. I wonder whether the split will affect the overall performance of OFA.

Hi @CurryTang , thanks for helping us improve our paper. We will update the code and results accordingly in our revision. Meanwhile, we conducted experiment using GCN and our split, the results seem to be improved a bit.

Hi @LechengKong, I have a follow-up question regarding the evaluation of OFA-ind-st. It seems there are throwing some errors when I try to evaluate the performance of link prediction with the following commands python3 run_cdm.py task_names pubmed_link d_multiple 1 d_min_ratio 1 lr 0.001 num_layers 3 num_epochs 2 dropout 0.15.
For Cora with the following setting,
cora_link: &cora_link
  <<: *E2E-link
  eval_set_constructs:
    - stage: train
      split_name: train
      dataset: cora_link
    - stage: valid
      split_name: valid
      dataset: cora_link_eval
    - stage: test
      split_name: test
      dataset: cora_link_eval
    - stage: test
      split_name: train
      dataset: cora_link

It throws ValueError: No samples to concatenate.
For Pubmed, it throws IndexError: list index out of range when conducting the validation.

For Pubmed, it should be
Traceback (most recent call last):
  File "/mnt/home/chenzh85/graphlang/PyGFM/MyOFA/run_cdm.py", line 219, in <module>
    main(params)
  File "/mnt/home/chenzh85/graphlang/PyGFM/MyOFA/run_cdm.py", line 167, in main
    val_res, test_res = lightning_fit(
  File "/mnt/home/chenzh85/graphlang/PyGFM/MyOFA/gp/lightning/training.py", line 66, in lightning_fit
    trainer.validate(model, datamodule=data_module, verbose=False)[0]
IndexError: list index out of range

For Cora, it should be
File "/mnt/home/chenzh85/graphlang/PyGFM/MyOFA/gp/lightning/metric.py", line 149, in eval_epoch
    return evlter.compute()
  File "/mnt/home/chenzh85/.local/lib/python3.10/site-packages/torchmetrics/metric.py", line 615, in wrapped_func
    value = _squeeze_if_scalar(compute(*args, **kwargs))
  File "/mnt/home/chenzh85/.local/lib/python3.10/site-packages/torchmetrics/classification/auroc.py", line 122, in compute
    state = (dim_zero_cat(self.preds), dim_zero_cat(self.target)) if self.thresholds is None else self.confmat
  File "/mnt/home/chenzh85/.local/lib/python3.10/site-packages/torchmetrics/utilities/data.py", line 34, in dim_zero_cat
    raise ValueError("No samples to concatenate")
ValueError: No samples to concatenate

I haven't changed the source code regarding the data loader and evaluation.

One side note is that it throws the warning
/python3.10/site-packages/lightning/pytorch/callbacks/model_checkpoint.py:383: ModelCheckpoint(monitor='valid_pubmed_link_eval/auc') could not find the monitored key in the returned metrics: ['train_eval/loss', 'epoch', 'step']. HINT: Did you call log('valid_pubmed_link_eval/auc', value) in the LightningModule?
when evaluating the link prediction, which is not shown when evaluating node classification.

Hi @CurryTang , I wasn't able to reproduce the error using your command on pubmed_link. I used your Cora setup and ran the same command as your pubmed_link command (replacing with cora_link), I also can't get the same error. However, we indeed had evaluation issues in our earlier versions, can you check if you are on the most recent commit and try again? Thanks.

Thanks for your response! I re-run the code on the new cloned version and it seems it works well. I will check my codebase to find out the problems.

Hi @LechengKong! I have a follow-up question regarding the setting of d_multiple and d_min_ratio. May you share some insights or rule-of-thumbs to set these values? I have tried using the ratio given in e2e_all_config.yaml, setting it according to the number of training samples, and also the logarithm of training samples. May you share how you tune these values to reduce the search space? Many thanks

Hi @CurryTang , that's a very good question! When we submitted the paper, we did not use an autonomous tool to tune the parameters, because there were just too many parameters for the datasets. However, we do have some insights on tuning these values. d_multiple is the initial frequency at which a dataset appears in one epoch. We have a mechanism that monitors the validation score, if the score is not improving for several epochs, we reduce the number of samples in one epoch by half. d_min_ratio controls the minimum number of samples in one epoch.
So the rule of thumb for tuning these parameters requires you to look into the training process. Because some datasets require more training time/epoch, and when these datasets become well-trained, the rest of the dataset might already be severely overfitted. So, you want to control d_multiple so that roughly all datasets get well-trained at the same time. For example, suppose you need to train arxiv dataset for 10 epochs, and each epoch contains 1000 data points so that it is well-trained (best validation score achieved). Meanwhile, to make the model well-trained on Cora, you need to train Cora for 2 epochs, each with  200 data points. So if you train 1000 arxiv + 200 Cora for 10 epochs, Cora will be overfitted due to the 8 epochs of extra training. Then, you can set d-multiple to alleviate this issue, you can set d_multiple for arxiv to be 1, and for cora to be 0.2, and after 10 epochs you sees 1 * 1000 * 10=10000 arxiv samples and 0.220010=400 cora samples, which is roughly the number of samples required to well-train both datasets.
This becomes complicated when you train all datasets together because it is not apparent what is the required total number of trained samples for each dataset. After all, both positive and negative transfers happen. What we did was that we started with all 1 d_multiple, and recorded when the validation accuracy hit the highest point. We take the number of samples trained at that moment as the optimal number of samples and adjust the d_multiple so that after the same number of training epochs, each dataset hits its optimal # of samples roughly at the same time. I am pretty sure there are more elegant curriculum learning methods that can do this properly, but the current approach seems to work out.

Thanks for your detailed reply!


possible mistake in NACA 4-digit series geometry creation

it is possible that Lines 131 to 132 of airfoils.py  should be changed to
p = float(naca_digits[1])/10
m = float(naca_digits[0])/100
to complain with NACA 4-digit series designation

Hi jsaldia,
I have looked into it and it looks like you were correct. I have implemented the changes in my version, but I want to discuss the results with Aaron before merging the code. If that can help I have compared the results with the ones obtained from http://www.airfoiltools.com/airfoil/naca4digit.
At the moment there is still a significant difference between the module and the website results (error of 1e-2), for extreme cases like a for the NACA9930 wing. Classical NACA profiles like the 00xx, 14xxx, 24xx and 64xx series should be precise enough to work with.

This is probably related to this.
When i create a NACA0002 with 125 points in x-direction (cos spacing), i get this (zoomed in on the LE):

But it should look like this:


Make pytest work without adding path

Ok, although I suppose the Makefile is quite useless when one can just invoke pytest. (Will be fixed by 8bf810c.)

It is providing the simple convenient entry path because many people know how to call "make" or "make test" regardless of the programming language specific ecosystem. Also we can easily extend the Makefile but the instructions stay the same :)