Fluent launch process is very slow giving warnings.

🔍 Before submitting the issue

[x] I have searched among the existing issues
[X] I am using a Python virtual environment

🐞 Description of the bug
When try to launch Fluent session, it takes a long time to get launched getting warnings messages regarding run-time issues.
Input:
import ansys.fluent.core as pyfluent
session = pyfluent.launch_fluent(show_gui=False)

Output:
pyfluent.launcher WARNING: Exception caught - RuntimeError: The launch process has been timed out.
pyfluent.launcher WARNING: Retrying Fluent launch with less robust command: C:\Program Files\ANSYS Inc\v241\fluent\ntbin\win64\fluent.exe 3ddp   -sifile="C:\Users\kgiannok\AppData\Local\Temp\serverinfo-zy71n9vz.txt" -nm -hidden

The session starts but the launching time is quite high.
I get the same behavior for pyFluent 0.16.dev0 - 0.19.dev1 versions and for Fluent 23R2 and 24R1.
I have no issues with v0.15.0.
📝 Steps to reproduce
import ansys.fluent.core as pyfluent
session = pyfluent.launch_fluent(show_gui=False)

💻 Which operating system are you using?
Windows
📀 Which ANSYS version are you using?
tested pyfluent versions  : 0.16.dev0 - 0.19.dev1
tested Fluent versions     : 23R2, 24R1
🐍 Which Python version are you using?
3.10
📦 Installed packages
accessible-pygments==0.0.4
alabaster==0.7.13
ansys-api-fluent==0.3.18
ansys-api-platform-instancemanagement==1.0.0
ansys-api-systemcoupling==0.1.0
ansys-dpf-core==0.9.0
ansys-dpf-gate==0.4.1
ansys-dpf-gatebin==0.4.1
ansys-fluent-core==0.17.0
ansys-grpc-dpf==0.8.1
ansys-platform-instancemanagement==1.1.2
ansys-sphinx-theme==0.12.3
-e git+https://github.com/pyansys/pysystem-coupling.git@82e50fe7c81a1d59c888df28a99760e4b64bb614#egg=ansys_systemcoupling_core
appdirs==1.4.4
asttokens==2.4.0
attrs==23.1.0
Babel==2.13.0
backcall==0.2.0
beautifulsoup4==4.12.2
black==23.10.0
bleach==6.1.0
cachetools==5.3.1
certifi==2023.7.22
charset-normalizer==3.3.0
click==8.1.7
colorama==0.4.6
comm==0.1.4
contourpy==1.1.1
cycler==0.12.1
decorator==5.1.1
defusedxml==0.7.1
docker==6.1.3
docutils==0.20.1
exceptiongroup==1.1.3
executing==2.0.0
fastjsonschema==2.18.1
fonttools==4.43.1
google-api-core==2.12.0
google-api-python-client==2.104.0
google-auth==2.23.3
google-auth-httplib2==0.1.1
googleapis-common-protos==1.61.0
grpcio==1.59.0
grpcio-health-checking==1.48.2
grpcio-status==1.48.2
h5py==3.10.0
httplib2==0.22.0
idna==3.4
imagesize==1.4.1
importlib-metadata==6.8.0
iniconfig==2.0.0
ipython==8.16.1
ipywidgets==8.1.1
isort==5.12.0
jedi==0.19.1
Jinja2==3.1.2
jsonschema==4.19.1
jsonschema-specifications==2023.7.1
jupyter-sphinx==0.4.0
jupyter_client==8.4.0
jupyter_core==5.4.0
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.9
kiwisolver==1.4.5
livereload==2.6.3
lxml==4.9.3
MarkupSafe==2.1.3
matplotlib==3.8.0
matplotlib-inline==0.1.6
mistune==3.0.2
mypy-extensions==1.0.0
nbclient==0.8.0
nbconvert==7.9.2
nbformat==5.9.2
numpy==1.26.1
numpydoc==1.6.0
packaging==23.2
pandas==2.1.1
pandocfilters==1.5.0
parso==0.8.3
pathspec==0.11.2
pickleshare==0.7.5
Pillow==10.1.0
platformdirs==3.11.0
pluggy==1.3.0
prompt-toolkit==3.0.39
protobuf==3.20.3
psutil==5.9.6
pure-eval==0.2.2
pyasn1==0.5.0
pyasn1-modules==0.3.0
pydata-sphinx-theme==0.14.1
Pygments==2.16.1
pypandoc==1.12
pyparsing==3.1.1
pytest==7.4.2
pytest-sphinx==0.5.0
python-dateutil==2.8.2
pytz==2023.3.post1
pywin32==306
PyYAML==6.0.1
pyzmq==25.1.1
referencing==0.30.2
requests==2.31.0
rpds-py==0.10.6
rsa==4.9
six==1.16.0
snowballstemmer==2.2.0
soupsieve==2.5
Sphinx==7.2.6
sphinx-autobuild==2021.3.14
sphinx-autodoc-typehints==1.24.0
sphinx-copybutton==0.5.2
sphinx-gallery==0.14.0
sphinx-notfound-page==1.0.0
sphinxcontrib-applehelp==1.0.7
sphinxcontrib-devhelp==1.0.5
sphinxcontrib-htmlhelp==2.0.4
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.6
sphinxcontrib-serializinghtml==1.1.9
sphinxcontrib-websupport==1.2.6
sphinxemoji==0.2.0
stack-data==0.6.3
tabulate==0.9.0
tinycss2==1.2.1
tomli==2.0.1
tornado==6.3.3
tqdm==4.66.1
traitlets==5.11.2
typing_extensions==4.8.0
tzdata==2023.3
uritemplate==4.1.1
urllib3==2.0.7
wcwidth==0.2.8
webencodings==0.5.1
websocket-client==1.6.4
widgetsnbextension==4.0.9
zipp==3.17.0

@giannoko
Could you please confirm once again with python 3.11.
It works fine for me with python 3.11 - '0.19.dev1' - 23R2 and 24R1.
Python 3.11.1 (tags/v3.11.1:a7a450f, Dec  6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ansys.fluent.core as pyfluent
>>> pyfluent.version_info()
'0.19.dev1'
>>> session = pyfluent.launch_fluent(show_gui=False)
>>>
>>> import os
>>> os.environ["PYFLUENT_FLUENT_ROOT"] = r"D:\Installations\Ansys\v232_14092023\ANSYS Inc\v232\fluent"
>>> session_232 = pyfluent.launch_fluent(show_gui=False)
>>>
>>>

Thank you.

The reason why it is slow is because your system for some reason fails to launch Fluent in the standard detached (and more robust) way, and that times out after 60 seconds by default when it notices Fluent isn't working. After timeout, it tries launching Fluent again, but now as a child of the Python process, which isn't good for multiple reasons, but mainly because it leaves Fluent vulnerable to bad termination attempts from IDEs such as Visual Studio (jupyter notebook "restart kernel" options and others like that) which may leave zombie processes behind that would need to be cleaned up manually.
As a quick workaround, can use the start_timeout argument (see launch_fluent documentation) to reduce the default timeout from 60 seconds to something like 10 seconds for example (don't make it too short or it will fail to launch Fluent in both attempts).
If Python 3.11 as Harshal suggested doesn't work, and neither does updating Windows, I am going to be in touch with @giannoko to try to debug this properly and see if we can figure out a better solution.

@hpohekar I get the same issue with python 3.11 also.


Implement function to print path

I am willing to take up this enhancement . Could you explain more about it. Thanks

We want to implement a function, i.e. print_path() to print the sequence of paths from source to destination with the shortest number of link, i.e. source url => url_1 => url_2 =>...=> destination url
For example:

from wikilink import WikiLink
app = WikiLink()
app.setup_db("mysql", "root", "12345", "<IP_ADDRESS>", "3306")
source = "https://en.wikipedia.org/wiki/Cristiano_Ronaldo"
destination = "https://en.wikipedia.org/wiki/Lionel_Messi"
app.min_link(source, destination, 6)
1
app.print_path()
/Cristiano_Ronaldo
/Lionel_Messi

I have submitted pull request . Please review

Thank you very much!
On Tue, 11 Jun 2019 at 15:43, Pradeep Nayak<EMAIL_ADDRESS>wrote:

I have submitted pull request . Please review
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/tranlyvu/wiki-link/issues/16?email_source=notifications&email_token=AC6MPI3F5UMJW7OXRYRT7LLPZ5JS5A5CNFSM4GMDHYFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXMHVJA#issuecomment-500726436,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AC6MPI2KJX3RGY5D5AG4JC3PZ5JS5ANCNFSM4GMDHYFA
.


NanoVG changes states, disrupts rendering

Hello.
I want to replace my current HUD with NanoVG. However, whenever I render directly (nvgBeginFrame) or with a nanovgFrameBuffer the rest of my scene does not display.
It seems to be an issue with state. if I make a single NanoVG call just once (after a 3 second delay for example) my scene remains blank indefinitely. Im now trying to determine what exactly nvgBeginFrame does to my environment.
Regardless of where I place the NanoVG code in my display function the rest of the scene simply doesn’t render at all. All I get is a white screen with the NanoVG elements. If I get rid of NanoVG calls within the begin/end frame functions my scene renders just fine. I've taken a look at the glnvg__renderFlush function and cant figure out which of the calls is causing this problem.
Furthermore I haven't been able to find any tutorial combining NanoVG and OpenGL render calls (i.e overlaying a 3D mesh). So its hard for me to determine what I’m doing wrong. Below is an example of my NanoVG call.
nvgBeginFrame(je.vg, je.gameWidth(), je.gameHeight(), 1.0f);
nvgSave(je.vg);
colorA.r(1).g(0).b(0).a(1);
nvgBeginPath(je.vg);
nvgRect(je.vg, 0, 0, 340, 30);
nvgFillColor(je.vg, colorA);
nvgFill(je.vg);
nvgRestore(je.vg);
nvgEndFrame(je.vg);
This is my current HUD. http://media.indiedb.com/images/downloads/1/82/81436/scndgen_legacy_2015-1-28-57903-s.png

The readme describes the state that is changed by nanovg. Are you restoring those after nanovg has rendered?

Hi. I've only restored glEnable(GL_BLEND); as that's the only flag I explicitly set at startup. Im not sure what the "defaults" are for the other states because I don't modify them.

nanovg tries to restore to some sane values. but for example textures and shaders are set to something else. the opengl reference manual has defaults for each state.

Thanks. Ill take a look into that.

Just in case anyone stumbles upon this in the future. A lot of initial values for various parameters are listed here.
https://www.khronos.org/opengles/sdk/docs/man/xhtml/glGet.xml

Hi memononen, I figured out the problem
#if NANOVG_GL_USE_UNIFORMBUFFER
enum GLNVGuniformBindings {
GLNVG_FRAG_BINDING = 0,
};
#endif
and
glUniformBlockBinding(... , GLNVG_FRAG_BINDING);
Im using GL4.4 and the block binding messed up my shaders persistent mappings (which were bound to block zero). I think it will be helpful to explicitly state that users should reserve block 0 going forward.
Thanks a lot for your help


control which services are started by the default CMD?

The currently documented configurations in  the README all seem to bring kibana up along with logstash and elasticsearch service.  It would be excellent to have some way of disabling kibana in particular on startup, as it seems to have a much higher memory footprint when the container is running, and may not always be needed.   (though I may be diagnosing this incorrectly, top just attributes it to java).
This is potentially related to #48, e.g. clearly this wouldn't be an issue if kibana wasn't installed in the image by default.

@cboettig are you suggesting I simply swap ENTRYPOINT out for CMD in the Dockerfile? That is on my todo list for the next release, 0.12.0.

Yup, that would do it, thanks
On Mon, Feb 16, 2015, 9:02 PM P. Barrett Little<EMAIL_ADDRESS>wrote:

@cboettig https://github.com/cboettig are you suggesting I simply swap
ENTRYPOINT out for CMD in the Dockerfile? That is on my todo list for the
next release, 0.12.0.
—
Reply to this email directly or view it on GitHub
https://github.com/pblittle/docker-logstash/issues/57#issuecomment-74617486
.

Mind hooking me up with a PR off of the develop branch? :smile:


The default/error behavior for the CSV input file is seldom reached

While working for #21, I've realized that these lines
https://github.com/Tux/CSV/blob/916c30e37fdd7db7900849e99c823f98444095cf/lib/Text/CSV.rakumod#L1792-L1794
Can only be reached if the input type does not subclass Any, which is extremely difficult to happen, because most classes but Junctions and Mu subclass it. That drops us into (possibly) unintended territory, because the default behavior for Any is to wait for standard input (which is what happens in #21, actually)

Nice analysis. This indeed is uncharted territory and has no counterpart in Perl5's Text::CSV_XS.
When I made the translation to Perl6 and later to Raku, I tried to implement all possible types that could even remotely be a source for CSV data. I am sure I missed some, and I am also sure I chose implementations that I would have done different when done today.
I'll just take your advice


Update jsDelivr links

jsDelivr switched to a fully automated system, that can serve files from npm and GitHub. This means all future releases will be available automatically, but will use a new link structure.
I updated the links now so you don't forget to do it when you release a new version.
You can find links for all files at https://www.jsdelivr.com/package/npm/yasgui-yasr.
Feel free to ping me if you have any questions regarding this change.

Thanks  @LukasDrgon , had no clue jsdelivr changed their API! I updated it in my build script instead (the link is dynamically updated based on the latest version)
Thanks for letting me know


Add legacy icons

Add legacy icons
🚀 Feature
f3fc77a - feat: add legacy icons from ovh-manager-webfont
🔖 TODOs

[x] remove duplicates.
[x] move from legacy to manager if icon is product oriented (must be decided later on).
[x] determine if we keep legacy- prefix in some cases. yes/no? yes
[x] move generic icons to their specific folders.
[x] remove deprecated ones if exists.
[x] rename icons by using kebabCase.

🗑 Icons removed (from ovh-manager-webfont)

filter
logo-ovh
search

You'll need to do a rebase ;)

@AxelPeter rebased done.


Map Editor (0.30.0): Fill in selected box only (Suggestion)

When using the new map editor in 0.30.0, I noticed the following rather unintuitive behaviour when trying to use the fill tool inside a selected box:
When you use the fill tool inside this selected box, it should only fill the area you selected:

However, the current effect is this:

The intuitive effect (as used in multiple other applications, like Paint.NET or Gimp) would be this:

Agreed. It would be very useful.

done in .30


difference between DIRECT7 、DIRECT1、and  KDTREE

thank you for your contribution, i don't know what is difference between DIRECT7 、DIRECT1 and KDTREE ？

For each point in the input cloud, NDT searches the neighbor voxels to calculate the gradient of the matching score. The original NDT implementation in PCL used kdtree-based radius search to find the neighbor voxels. But, the radius search is expensive processing, and it affects the matching speed. DIRECT1 and DIRECT7 directly finds the neighbor voxels by indexing. As illustrated in the figure, DIRECT1 just returns the voxel which contains the point (voxel[x, y]), while DIRECT7 extracts the seven voxels next to the center voxel (voxel[x, y], voxel[x-1, y], voxel[x, y-1], ...).

The matching result with KDTREE is identical to the original version in PCL, and it is slightly faster thanks to the SSE optimization. DIRECT1 is much faster, but a bit unstable since it refers narrow area compared to KDTREE. DIRECT7 is faster than KDTREE, and it extracts the same voxels as KDREE in most of cases.

Hi @bob-ROS ,
We've not published some papers about the detail of this implementation. If you need general explanation of the NDT algorithm, this thesis would be good to refer.
This implementation was developed as a part of the following work, and I would be glad if you refer to it in case it is appropriate for your paper.

Kenji Koide, Jun Miura, and Emanuele Menegatti, A Portable 3D LIDAR-based System for Long-term and Wide-area People Behavior Measurement, International Journal of Advanced Robotic Systems, 2019


How to mark entities in markdown format in case of ner_crf containers

Rasa NLU version:11.4
Operating system (windows, osx, ...):win 10
Content of model configuration file:

Issue:
My json data looks like this
{     "text": "1000",     "intent": "press",     "entities": [       {         "start": 0,         "end": 4,         "value": "1000",         "entity": "emp_id"       }     ]   }
Is there anyway to write it in markdown format.(ner_crf)
I think it must be along lines of
[1000](text:emp_id)

Please ask these kind of questions in our forum
And here's the relevant documenation: https://rasa.com/docs/nlu/dataformat/


Support policy documents

Fixes #1932

This change is

@remyabel Argh, I did not notice the problem with join() vs. Join(), good thing we have tests, can you fix it?


Custom draggable nodes

This is awesome! I love how straightforward tracking the nodes is, but I am wondering what the best way to create my own node types/templates i.e. "Slack Message", "Github Star", "Template", etc. I've been looking through both the css and js and I don't see any references pointing to those elements in the side-menu in your demo yet. Maybe i'm missing something though?

i did not notice the case statement in index.html - looks pretty self explanatory


Localizations / Maintenance update

Hello,
I think it would be something great to add localizations to the app, I really miss it (French).
Plus, I think there should be a maintenance update, or some features added or ergonomic enhancements, as it is quite "repulsive" to see an app that has been last update in 2016! (I'm not saying that you're trash, hu, just telling that maybe, just maybe, a little maintenance won't hurt the app!
Thanks for reading guys, keep up the good work!

Thanks for your feedback. I think you may have intended it for our currently live app at https://github.com/duckduckgo/ios-search-and-stories. The good news is that this repo contains the prototype of a new private browsing app that we're building. We haven't added localization to this one yet but will in the future.


feat: unramified, smooth, etale are stable under base change

Thanks
maintainer merge

Thanks!
bors merge


[CP] Merge into [beta] channel: Bad identifier for async closure generator body names in extension methods.

commit(s) to merge: a38e265f04d8976d226c3d966de2d9070a5c0941
merge instructions: clean merge
what's the problem:
Bad JavaScript code generation when extension methods are used with generators (async, async*, sync*).
why cherry-pick:

The issue has come up twice already. #42531 and #42712.
Low impact - it is a one-line change (plus new tests).

@rakudrama Can you update the request to look something like https://github.com/dart-lang/sdk/issues/42715

Note:  this issue is more visible in 2.9. The problem is exposed more often in the presence of legacy types, and since 2.9.* is the first release with the unforked SDK, we are noticing this issue a lot more than before.

lgtm

risk seems low
customers are hitting this
per @sigmundch 's last comment, it's effectively a regression

Approved

Cherry-picked to 2.9.0-21.10.beta.


Sane API for redux store part 1

so, it's a lot of different efforts
I see the final result to be the tests written the way to highlight the client API

The latest changes are in test folder.

We still need to figure out how this thing's gonna be returning the application state that we later gonna be feeding to Provider
We write a something-redux application after all)))


[Bug] Windows 11 - WMIC not enabled by default

Describe the bug
After installing vpin studio server on a new windows 11 machine, the software just hung up at the splash screen.
after reviewing the server log, the log indicated that wmic could not be run.
16-10 09:25:10.199 WARN  [/api/v1/system/info]d.m.v.r.u.SystemUtil - Failed to resolve cpu id: Cannot run program "wmic": CreateProcess error=2, The system cannot find the file specified
After investigation, it appears that newer installations of windows 11 have wmic disabled by default.
no error messages aside from the server log indicated the problem.
After enabling wmic in Optional Features, everything is running correctly.
I hope this helps someone else
C

Funny coincidence. I got my new Window 11 PC from work and noticed the same thing yesterday.
Patch will go out this weekend.

great news. hopefully this is an easy one for fix.
Cheers.

Fixed with 3.8


org.bitcoinj.crypto.KeyCrypterException: Could not decrypt: pad block corrupted

New error report uploaded. Id: 2ffaac2
MultiBit HD version: 0.1.2
Operating system: 64 6.1 Windows 7
User notes length: 9
Number of stack traces: 9
First stack trace:
org.bitcoinj.crypto.KeyCrypterException: Could not decrypt: pad block corrupted
at org.multibit.commons.crypto.AESUtils.decrypt(AESUtils.java:127) ~[multibit-hd.jar:na]
at org.multibit.hd.core.managers.WalletManager.loadWalletFromFile(WalletManager.java:954) [multibit-hd.jar:na]
at org.multibit.hd.core.managers.WalletManager.loadFromWalletDirectory(WalletManager.java:1068) [multibit-hd.jar:na]
at

This looks like a failed password - did this trigger the error or was it later?

Agreed most likely a wrong password but it needs checking if the Error Reporter was triggered (which is incorrect) or not.

Just tried various incorrect passwords and:

the UI behaved as expected (show message, enable back button)
there were no stack traces in the log that would get picked up be the ErrorReporter
the ErrorReporter never fired.

As this is a 0.1.2 bug I'll close it as 'could not reproduce'


Add specific information to README.md for first milestone

Context
During the first milestone , theres a demo we need to do about the first 10% of the work.
Goals
Adapt the README.md file in a way that is easy for reviewers to find the all the information and the intentionality of the demo.

Hello @geofmureithi @mariopil , here is the PR ready for review for the demo readme. Let me know your thoughts :)
https://github.com/eigerco/nebula/pull/16


Failed to run Flake8-sql

Hey @pgjones, How do I run this plugin so that Linting can take place fo specifically SQL?
When I run using flake8 and put the rule e.g Q440 linting does not take place.
I have tried and searched around but failed to get the correct commands.
Can you please update the readme for guidance on this issue?

This linter will run over the files you run flake8 over there is nothing special to do after installing. This documentation may help.

I'm troubleshooting a different issue and made a quick test file that might be useful. Save this snippet as flake8_sql_check.py
"""Test of flake8-sql.

Test with: python -m flake8 flake8_sql_check.py

Which should output at a minimum:

flake8_sql_check.py:11:9: Q443 incorrect whitespace around comma
flake8_sql_check.py:12:9: Q443 incorrect whitespace around comma
flake8_sql_check.py:14:9: Q444 incorrect whitespace around equals
flake8_sql_check.py:15:9: Q444 incorrect whitespace around equals

If needed, check installed extensions with: python -m flake8 --bug-report

"""

# https://github.com/pgjones/flake8-sql/blob/0ed4d53dfed6d243254d9065646b17cf8f49708d/tests/test_cases/whitespace.py
query = "SELECT ca,cb FROM tbl"  # Q443
query = "SELECT ca ,cb FROM tbl"  # Q443
query = "SELECT ca, cb FROM tbl"
query = "SELECT ca FROM tbl WHERE ca= 'b'"  # Q444
query = "SELECT ca FROM tbl WHERE ca ='b'"  # Q444
query = "SELECT ca FROM tbl WHERE ca = 'b'"
query = "SELECT ca FROM tbl WHERE ca != 'b'"


AndroidX Jetpack migration needed

Well, I think we need to publish this library with AndroidX Jetpack ASAP. I checked so many dependencies on my project, and found that 3 of 19 dependencies are still using old Support Library. Material Range Bar is one of them. I am aware that this lib will cause crash to my app since Support Library has different package name with AndroidX Jetpack. We don't have to wait until stable version 1.0.0 is released by Google. We can do it now. I have tested AndroidX and found no problem with it.

@TurKurT656 have you tried

enableJetifier
flag?

@TurKurT656 have you tried enableJetifier flag?

yes it was enabled.

@anggrayudi @TurKurT656 Could you provide more info on the issue? Crash log, material-range-bar version, android gradle plugin/build tools  version and so on
Particularly I use the library (1.4.5) along with androidx  appcompat 1.0.2 and android gradle plugin 3.3.1 and default build tools version for this plugin.

android.enableJetifier=true
android.useAndroidX=true

are enabled too. I have tested our app on few android versions 18 to 28 and found no issues so far.
However, it is still vital for me know about any problems I could have missed before releasing the app to prod.
Could you confirm the library is crashing your apps even using the config provided by me?

@anggrayudi @TurKurT656 Could you provide more info on the issue? Crash log, material-range-bar version, android gradle plugin/build tools version and so on
Particularly I use the library (1.4.5) along with androidx appcompat 1.0.2 and android gradle plugin 3.3.1 and default build tools version for this plugin.

android.enableJetifier=true
android.useAndroidX=true

are enabled too. I have tested our app on few android versions 18 to 28 and found no issues so far.
However, it is still vital for me know about any problems I could have missed before releasing the app to prod.
Could you confirm the library is crashing your apps even using the config provided by me?

Today I've tested the newer version (1.4.5) with androidX and there is no error.


[DISCUSSION] Deltastreamer - Reading commit checkpoint from Kafka instead of latest Hoodie commit

Our requirement is actually this: Supporting multiple deltastreamers writing to a single hudi table
https://github.com/apache/hudi/issues/6718
HUDI-5077
We reviewed the deltastreamer code and noticed that the deltastreamer can read commits from Kafka consumer groups if it doesn't find the last checkpoint from Hoodie.
We're considering modifying the code to always read commits from Kafka, based on a new flag, by making changes in the Hudi deltastreamer.
Do you foresee any nuances or issues with this approach?
Environment Description

Hudi version : 0.13.0

Spark version : 3.3.2

Storage (HDFS/S3/GCS..) : S3

Running on Docker? (yes/no) : no

We reviewed the deltastreamer code and noticed that the deltastreamer can read commits from Kafka consumer groups

If the consumer offset is what the Hoodie checkpoint persisted, sounds more reasonable to always pull the offset from the Kafka server where the offset can be managed in good shape.

@danny0405 : We made changes to hudi, and will be using the code internally. The following PR embodies the proposed changes. Any thoughts or inputs? Will be happy to work more on getting this PR merged to hudi.
https://github.com/apache/hudi/pull/11383

Ususally we do not creating PR agains released tag, can you fire a new one against master?

@danny0405 : In master i.e. HUDI version 1.0.0, there is a flag --ignore-checkpoint, which will do the same thing. We use version 0.13.0 for now, so would use this for now till 1.0.0 is stable and we are ready to upgrade.
But by default, when auto.offset.reset = group, and for the first attempt when there are no commits in kafka, it defaults to latest, which might mean loss of data. Either that should be earliest, or there should be an option to choose earliest to prevent missing any data.


Wrong user flow involving analyses and empty layers

Context
Right now, if you add certain analysis to a layer that doesn't return any data, Builder considers that layer as empty. And, being an empty layer, it disables all the other tabs, but Data. So users are unable to access the Analysis tab to edit or delete said analyses.
There's a workaround, by accessing them through the Layers list' analysis nodes "shortcuts" but users should be able to access them from both sides.

Steps to Reproduce

Apply an analysis in a layer that will filter out all the data in that layer.
Go back to the Layers list.
Access the layer again.

Expected result
In that instance, when an analysis is filtering out all the data, we shouldn't consider the layer as empty, but as a filtered.
This is an important one @ramiroaznar

@arianaescobar Should we take the user to the analysis tab when the layer is empty and has any analysis applied? I think we should, right?

It shouldn't be a particular new state, it should be the default state with everything enabled.

Should we enable Style, Pop-up, and Legends tabs too? That behaviour of disabling these three tabs wasn't modified when we disabled the analysis tab in the empty layer state, it was changed before.

Everything has to be enabled. This isn't an empty state, it's a default normal state with inputs from the user on the analysis tab.
Disabling those tabs was never meant for this instance.

@matallo and I have been talking about this, and even though the tabs should be enabled when layer is empty because of an analysis filtering, it requires further action that should be addressed in other issue.
These actions involve Style, Popup and Legend tabs.
In the Style tab, the style shouldn't be reset when the layer is empty to restore the previous state after reverting the filter if the layer geometry and querySchemaModel has the same schema.
In the Popup tab, we shouldn't let the user select any column when the layer is empty.

In the Legends tab, we shouldn't let the user modify the current legend because we don't have the current layer styles, so there won't be any items in the legend.
In order not to block the important fix for more time, what do you think about addressing the thing of the disabled tabs in another issue?

@arianaescobar It is now deployed in ded11.

I don't think we should address it in a separate issue because of interactions like the following two (take a look at how the tabs change all of the sudden in both):

We shouldn't merge it to production like this :(

Yes, you are totally right. I am looking at those cases to fix them ASAP.

I have just uploaded a new revision to ded11 staging. The two bugs that @arianaescobar mentioned should be fixed.
The tabs have been enabled when the layer is empty and has any analyses. And the second bug, which was a routing bug, should be resolved as well.

@arianaescobar @jesusbotella I have checked and this is also happening when applying a query that does not show any geometry (for instance not adding cartodb_id or the_geom_webmercator). That needs to be fixed as well.

A few other things...
We're still showing the layer as if it were an "Empty layer" and it is not. This icon shouldn't appear.

If we go to the DATA tab, we should this message from the Empty layer state but in this case, it doesn't apply. Users can't even use the Add geometry feature.

Let's change it do this:

No data available.
There are no results for the combination of analyses applied to your data. Try tweaking them.

It might be a good idea to also show this message in all the other tabs –but Analysis–.

I've already added this message in the "Data" view.

It might be a good idea to also show this message in all the other tabs –but Analysis–.

Do you mean the same message? Or instead of "No data available" write "No style available"? (and the same with infowindows and legends) cc @arianaescobar

Awesome, Elena!
The same exact one! Without data, you can't style or add pop-ups nor legends. So that's step 1.

Got it! Thanks!

Is this issue solved then?

backend tests not passing, ASAP they do we're merging

Cool! I thought it was a RT issue, but I see Bravo is taking care of it.

The layer is empty (no rows in the dataset)

Only data tab enabled.
Text: This layer is empty
Warning icon should be visible both in layer list and in layer edition panels.

The layer is filtered (there are rows but an analysis returns 0 of them)

Warning icon should NOT be displayed in any place.
Analysis tab should be accesible.
Data, Style, Popup and legend should show the following text.

Layer that needs geocoding

What text in data tab?
What tabs are enabled?
Should we display the warning icon?

What text in data tab?

"There is no geometry in your data.
Apply a Geocode analysis or manually add geometry data to visualize on the map."

What tabs are enabled?

Only DATA and ANALYSIS.

Should we display the warning icon?

Yes, but only inside the layer.
Tooltip: "Geocode to visualize on the map"

🚀 Shipped!


Output Model Weights, Graph Files

Hi @StephenChan,
Just curious to see if there would be a way to obtain the weights / graph files of a trained model for offline use?
Thanks,
Jordan

Hi Jordan - no there currently isn't.
The download of the trained model (classifier) itself is certainly feasible, as it's a pickled CalibratedClassifierCV on the order of 5-10 MB large (about the same as a typical image). So far we've been able to make the unpickling work across different scikit-learn versions.
One wrinkle off the top of my head: I'm not sure how strictly the image-features format (the input into predict_proba()) needs to match CoralNet's in order to get meaningful results from the classifiers. If it's too strict, then maybe one has to get our pyspacer module up and running to meaningfully use the downloaded classifiers, at which point you'd be able to train your own classifiers offline anyway.
But either way, I think it wouldn't hurt to implement this sometime. We've been looking into providing more info/transparency about previous-best classifiers and classifier details in general, which may include each individual classifier getting its own details page. Model downloads would fit well with that idea.

Oh, so at what point in the model pipeline are you using Scikit-Learn? I thought y'all switched over to EfficientNetB0? Or, do you have a fine-tuned (on coral images) EfficientNetB0 encoder, and just use it as a feature extractor, which outputs into an individual Scikit-Learn model for each source?
Regardless, yeah having the pre-processing code for input images would be necessary otherwise the results of the model would be garbage. Some deep learning frameworks do allow you to have layers that perform pre-processing within the actual model architecture, so you might be able to package everything together?
I just bring it up because, the API is super useful (I'm working on creating a GUI and .exe, see below), but if a project had computation resources to perform inference themselves, but wanted to continue using the model trained on CoralNet, it might be useful for downstream tasks that require faster inference, or no access to internet.

Specifically, Coralnet  uses the EfficientNet architecture, but the network
was trained on Benthic images. For the patch surrounding an image point,
the network is used to compute a feature vector, which is then classified
using a linear classifier or multilayer perceptron (scikitlearn). The
EfficientNet feature extractor is used across all CoralNet sources. The
classifier is trained (and retrained) for each source. As more images are
labelled/confirmed, the classifier is retrained.
While we save the classifier on CoralNet, we have not created a way to
export the classifier or use the weights outside of CoralNet and the API.
David
On Thu, Jun 29, 2023 at 5:23 PM Jordan Pierce @.***>
wrote:

Oh, so at what point in the model pipeline are you using Scikit-Learn? I
thought y'all switched over to EfficientNetB0? Or, do you have a fine-tuned
(on coral images) EfficientNetB0 encoder, and just use it as a feature
extractor, which outputs into an individual Scikit-Learn model for each
source?
Regardless, yeah having the pre-processing code for input images would be
necessary otherwise the results of the model would be garbage. Some deep
learning frameworks do allow you to have layers that perform pre-processing
within the actual model architecture, so you might be able to package
everything together?
I just bring it up because, the API is super useful (I'm working on
creating a GUI and .exe, see below), but if a project had computation
resources to perform inference themselves, but wanted to continue using the
model trained on CoralNet, it might be useful for downstream tasks that
require faster inference, or no access to internet.
[image: Gooey_3]
https://urldefense.com/v3/__https://user-images.githubusercontent.com/115024024/249900038-0b04ccae-ecb3-4d58-9420-7267c444245b.JPG__;!!Mih3wA!HjpkdS4bHoxr9ciH7q9ZaEfRKbgZZmAfvFVaEms_fvdiI-zJll0T6dfYOBbTGWuZ1GAiMKs1No4i9cv30c_A43OH$
[image: Gooey_4]
https://urldefense.com/v3/__https://user-images.githubusercontent.com/115024024/249900093-2f613de9-64b4-4e7c-b182-48ad6655ba27.JPG__;!!Mih3wA!HjpkdS4bHoxr9ciH7q9ZaEfRKbgZZmAfvFVaEms_fvdiI-zJll0T6dfYOBbTGWuZ1GAiMKs1No4i9cv30WhB903m$
—
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/beijbom/coralnet/issues/481*issuecomment-1613827123__;Iw!!Mih3wA!HjpkdS4bHoxr9ciH7q9ZaEfRKbgZZmAfvFVaEms_fvdiI-zJll0T6dfYOBbTGWuZ1GAiMKs1No4i9cv30fJV5vMH$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABKA5AMKEHIICJUHESTU5UDXNXW5VANCNFSM6AAAAAAZVYUX3M__;!!Mih3wA!HjpkdS4bHoxr9ciH7q9ZaEfRKbgZZmAfvFVaEms_fvdiI-zJll0T6dfYOBbTGWuZ1GAiMKs1No4i9cv30T0SZa3a$
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Nice GUI! Interesting to see how folks are getting the most out of the API.
For what it's worth - in our pipeline, inference on features is a fair bit faster than feature extraction. ~5 times faster at least, as a back of the envelope number.


Is the project still active?

Hi!
I came across this project today and found very interesting. I wonder, however, if it still active since the last commit was months ago and the website is now offline.
I'm relatively new to JS but I sure would like to help.
Thank you

Hi @fbidu ,
Yes, at this moment the project is offline and I would really glad if someone can take ownership and resume it again. :+1:


No module named 'pygribjump'

What happened?
Hi,
  This issue is more of a question than a bug report.

 A couple of month ago someone you probably know (Dr. Quintino) gave a video conference where I learned about "polytope", earthkit, ... etc.  A couple of slides showed performance results (timings of feature extraction using polytope and GribJump).  It prompted me to install the package with conda and play with a few examples. One of them was 3D_shipping_route_wave_model.py  making use of the FDB backend ...

python 3D_shipping_route_wave_model.py
Traceback (most recent call last):
File ".../3D_shipping_route_wave_model.py", line 8, in
from polytope.datacube.backends.fdb import FDBDatacube
File .../python3.10/site-packages/polytope/datacube/backends/fdb.py", line 4, in
import pygribjump as pygj
ModuleNotFoundError: No module named 'pygribjump'
I searched for pygribjump ... it was nowhere to be found.
I realize that the software is a work in progress.  I just want to check with you that the module  (GribJump, pygribjump)
is indeed not publicly available and the outcome is as expected ?
I look forward to see more performance results with earthkit/polytope. Your work is interesting and appears to be really tigthly
integrated to the software infrastructure at ecmwf. This is new stuff in comparison to what is done in the context of PANGEO
for example.
Anyway, thanks for your time,
Oh, and I did succeed in running some other tests though ... :-)
What are the steps to reproduce the bug?
none
Version
polytope 1.03
Platform (OS and architecture)
cat /etc/os-release --> Ubuntu 22.04.4 LTS     uname -a --->  Linux 4.18.0-240.el8.x86_64
Relevant log output
none

Accompanying data
none
Organisation
Environment and Climate Change Canada

Hi @jmb64,
You are quite right, we had not made gribjump public yet, but now we have :)
https://github.com/ecmwf/gribjump
It is very experimental and we are actively developing this whole stack. To use gribjump you'll need to build it, along with FDB (https://github.com/ecmwf/fdb) and it's dependencies. pygribjump needs to be pip installed directly from the git repository too, as its not on pypi yet.
To be honest the polytope-gribjump-fdb stack is not really in a state where we'd expect people to be able to deploy it succesfully, but you're welcome to try!

Hi James,
Thanks for your reply.  I won't loose any sleep over it but I will probably try ... :-)

It seems that access to the python api pygribjump is rectricted though.
Here's what I get:

python3 -m pip install --upgrade git+ssh://git@github.com/ecmwf/gribjump.git@master
Collecting git+ssh://@github.com/ecmwf/gribjump.git@master
Cloning<EMAIL_ADDRESS>(to revision master) to /tmp/jmb001/203/pip-req-build-csew7wed
Running command git clone --filter=blob:none --quiet<EMAIL_ADDRESS>/tmp/jmb001/203/pip-req-build-csew7wed
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
<EMAIL_ADDRESS>Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.

Right on. Thanks!

Would you happen to have an example showing the content for this file ?
GRIBJUMP_CONFIG_FILE = "/path/to/gribjump/config.yaml"

Hi @jmb64 ,
The config for GribJump is very minimal (for now), and in principle should be optional. For a local extraction on your machine, the following config.yaml should do the trick:
---
type: local

Hi,
Ok, I probably got as far as i could without having an actual instance of FDB installed locally.  I thought I could maybe get away with plain grib file(s) on GPFS to mimick FDB (with indexes on the side ?). By the way how do you let your python scripts know  the location of your local FDB ?    As a suggestion, I don't know how complex this would be but  ... a simple "example FDB" setup would be useful for running tests outside the walls of ECMWF I guess. Apart from that I think my software installation appears to be ok. A few scripts are not completing successfully (polytope requests definition) but I suspect it is because they are outdated w/r to the api. This is on me. It is a work in progress. I was warned and that's fine :-)

I will follow your development and docs .
Thanks.
Jean-Marc

Hi Chris,
 Just to let you know that I managed to get things a little bit more "concrete" for myself with this FDB ... using

readthedocs.io (fdb and earthkit-data). To answer my own question about an FDB example for testing ... i was able to create
the POSIX filesystem flavor of FDB from grib through earthkit. I also noticed the grib2fdb cli tool in my build of the whole shebang ... The hard part is not knowing much about the ECMWF culture (MARS language ...)
One last thing, my fdb build was incomplete (cli tools and tests/api...) and I solved it by adding  libcurl (-lcurl) to a bunch
of link.txt files. I don't know if it was something I missed in the configuration (ecbuild) or if it should be corrected in fdb github ...

Maybe one last question ...
Running this test
https://github.com/ecmwf/polytope/blob/develop/tests/test_fdb_datacube.py
it seems I cannot get "step" to make it as an FDB datacube axes.
This is the bit of code which I believe should create the datacube instance:

self.options = {
"config": [
{"axis_name": "step", "transformations": [{"name": "type_change", "type": "int"}]},
{"axis_name": "number", "transformations": [{"name": "type_change", "type": "int"}]},
{
"axis_name": "date",
"transformations": [{"name": "merge", "other_axis": "time", "linkers": ["T", "00"]}],
},
{
"axis_name": "values",
"transformations": [
{"name": "mapper", "type": "octahedral", "resolution": 1280, "axes": ["latitude", "longitude"]}
],
},
{"axis_name": "latitude", "transformations": [{"name": "reverse", "is_reverse": True}]},
{"axis_name": "longitude", "transformations": [{"name": "cyclic", "range": [0, 360]}]},
]
}
self.config = {"class": "od", "expver": "0001", "levtype": "sfc", "stream": "oper"}
self.fdbdatacube = FDBDatacube(self.config, axis_options=self.options)
Traceback (most recent call last):
File "...ECMWF/EARTHKIT_POLYTOPE/test_fdb_datacube.py", line 102, in
a.test_fdb_datacube()
File "... ECMWF/EARTHKIT_POLYTOPE/test_fdb_datacube.py", line 78, in test_fdb_datacube
result = self.API.retrieve(request)
File ".../.conda/envs/ecmwf/lib/python3.10/site-packages/polytope/polytope.py", line 58, in retrieve
request_tree = self.engine.extract(self.datacube, request.polytopes())
File ".../.conda/envs/ecmwf/lib/python3.10/site-packages/polytope/engine/hullslicer.py", line 140, in extract
self._unique_continuous_points(p, datacube)
File ".../.conda/envs/ecmwf/lib/python3.10/site-packages/polytope/engine/hullslicer.py", line 27, in _unique_continuous_points
mapper = datacube.get_mapper(ax)
File ".../.conda/envs/ecmwf/lib/python3.10/site-packages/polytope/datacube/backends/datacube.py", line 128, in get_mapper
return self._axes[axis]
KeyError: 'step'
I get this KeyError during the extraction ... which I figure is because "step" has not been added to axes ?
I added a few extra print  ... in the
JMB: datacube.py create_axes_config: {'config': [{'axis_name': 'step', 'transformations': [{'name': 'type_change', 'type': 'int'}]}, {'axis_name': 'number', 'transformations': [{'name': 'type_change', 'type': 'int'}]}, {'axis_name': 'date', 'transformations': [{'name': 'merge', 'other_axis': 'time', 'linkers': ['T', '00']}]}, {'axis_name': 'values', 'transformations': [{'name': 'mapper', 'type': 'octahedral', 'resolution': 1280, 'axes': ['latitude', 'longitude']}]}, {'axis_name': 'latitude', 'transformations': [{'name': 'reverse', 'is_reverse': True}]}, {'axis_name': 'longitude', 'transformations': [{'name': 'cyclic', 'range': [0, 360]}]}]} [{'axis_name': 'step', 'transformations': [{'name': 'type_change', 'type': 'int'}]}, {'axis_name': 'number', 'transformations': [{'name': 'type_change', 'type': 'int'}]}, {'axis_name': 'date', 'transformations': [{'name': 'merge', 'other_axis': 'time', 'linkers': ['T', '00']}]}, {'axis_name': 'values', 'transformations': [{'name': 'mapper', 'type': 'octahedral', 'resolution': 1280, 'axes': ['latitude', 'longitude']}]}, {'axis_name': 'latitude', 'transformations': [{'name': 'reverse', 'is_reverse': True}]}, {'axis_name': 'longitude', 'transformations': [{'name': 'cyclic', 'range': [0, 360]}]}]
JMB: fdb.py self.fdb_coordinates.items()  dict_items([('values', [])])
JMB: fdb.py name_values  values []
JMB: opt  axis_name='values' transformations=[MapperConfig(name='mapper', type='octahedral', resolution=1280, axes=['latitude', 'longitude'], local=None)]
JMB: self._axes  {'latitude': <polytope.datacube.datacube_axis.FloatDatacubeAxis object at 0x1517b7e39c30>, 'longitude': <polytope.datacube.datacube_axis.FloatDatacubeAxis object at 0x1517b7e39cc0>}
JMB get_mapper self._axes= {'latitude': <polytope.datacube.datacube_axis.FloatDatacubeAxis object at 0x1517b7e39c30>, 'longitude': <polytope.datacube.datacube_axis.FloatDatacubeAxis object at 0x1517b7e39cc0>}
I believe latitude and longitude have been added to the datacube axes from these prints ...
Thanks.
JM

Logging from polytope for
python test_fdb_datacube.py:
cat lis_test_fdb_datacube.log
INFO:root:Created an FDB datacube with options: {'config': [{'axis_name': 'step', 'transformations': [{'name': 'type_change', 'type': 'int'}]}, {'axis_name': 'number', 'transformations': [{'name': 'type_change', 'type': 'int'}]}, {'axis_name': 'date', 'transformations': [{'name': 'merge', 'other_axis': 'time', 'linkers': ['T', '00']}]}, {'axis_name': 'values', 'transformations': [{'name': 'mapper', 'type': 'octahedral', 'resolution': 1280, 'axes': ['latitude', 'longitude']}]}, {'axis_name': 'latitude', 'transformations': [{'name': 'reverse', 'is_reverse': True}]}, {'axis_name': 'longitude', 'transformations': [{'name': 'cyclic', 'range': [0, 360]}]}]}
DEBUG:root:Skipping /etc/polytope/config.json, file not found.
DEBUG:root:Skipping /etc/polytope/config.yaml, file not found.
DEBUG:root:Skipping /home/jmb001/.polytope.json, file not found.
DEBUG:root:Skipping /home/jmb001/.polytope.yaml, file not found.
DEBUG:root:Conflated config: {}
INFO:root:Axes returned from GribJump are: {}
INFO:root:Polytope created axes for: dict_keys(['latitude', 'longitude'])

Hi Mathilde,
       I appreciate you taking the time to offer these explanations. I kinda realize friday that actual data (FDB) was playing a

major role in the datacube configuration :-) ... just by changing the FDB I noticed I was getting past the configuration (i.e.
step axes ...) issue . I tried retrieving a GRIB file output from the HRES WAM with the 2d spectrum (2dfd) ... but it is not
available it seems (damn! :-)) ... as you mentioned. I will see if I can rewrite an example akin to 3D_shipping_route_wave_model but with available data this time ! I will re-read your post carefully, i'm sure this will help, and might have some questions later ...
Regards,
Jean-Marc
PS: You probably know the answer to this one ...  In Tiago's presentation (Bytes from Petabytes, slide 20) there's a graph showing linear scaling for the extraction process for 1000 to 4500 values on Lustre ...  we're talking about double precision values right ? Hence the 36kB(4500 values)/730ms ---> 4-6 ms/value ... Are those results from /examples or /performance on github ?

Hello again,
 I decided to adapt the polytope/FDB example "3D_shipping_route_wave_model.py", creating an FDB from era5 specific humidity in the marine boundary layer since I don't have access to the wave modele data (let,s say one is interested in shipborne lidar data , .. i.e. HALO/Wales)  using this CDS request:

#!/usr/bin/env python
import cdsapi
c = cdsapi.Client()
c.retrieve("reanalysis-era5-complete", {
"class": "ea",
"date": "2021-07-01/to/2021-07-31",
"expver": "1",
"levelist": "100/101/102/103/104/105/106/107/108/109/110/111/112/113/114/115/116/117/118/119/120/121/122/123/124/125/126/127/128/129/130/131/132/133/134/135/136/137",
"levtype": "ml",
"param": "133",
"step": "0/1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18",
"stream": "oper",
"time": "18:00:00",
"type": "fc",
"grid": "0.25/0.25"
}, "era5_202107_regular_ll_2.grib")
then create the local FDB

fdb write era5_202107_regular_ll_2.grib. I keep the same shapefile for shipping route (Shipping-Lanes-v1.shp)

Ruuning the modified script for specific humidity the extraction process encounters a problem in GribJump:

GRIBJUMP_DEBUG Using FDB's (new) axes impl
GRIBJUMP_DEBUG Base request: retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=1,stream=oper,type=fc
GRIBJUMP_DEBUG   Flattened request: retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=1,stream=oper,type=fc
GRIBJUMP_DEBUG Flattened keys for request retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=1,stream=oper,type=fc: [class=ea,date=20210702,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=1,stream=oper,time=1800,type=fc]
GRIBJUMP_DEBUG Base request: retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=2,stream=oper,type=fc
GRIBJUMP_DEBUG   Flattened request: retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=2,stream=oper,type=fc
GRIBJUMP_DEBUG Flattened keys for request retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=2,stream=oper,type=fc: [class=ea,date=20210702,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=2,stream=oper,time=1800,type=fc]
GRIBJUMP_DEBUG Base request: retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=3,stream=oper,type=fc
GRIBJUMP_DEBUG   Flattened request: retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=3,stream=oper,type=fc
GRIBJUMP_DEBUG Flattened keys for request retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=3,stream=oper,type=fc: [class=ea,date=20210702,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=3,stream=oper,time=1800,type=fc]
GRIBJUMP_DEBUG Built flat keys
GRIBJUMP_DEBUG Built keyToExtractionItem
GRIBJUMP_DEBUG Union request retrieve,class=ea,date=20210702,time=1800,domain=g,expver=0001,levelist=137,levtype=ml,param=133,step=1/2/3,stream=oper,type=fc
GRIBJUMP_DEBUG Found 3 fields in 1 files
GRIBJUMP_DEBUG File map:
GRIBJUMP_DEBUG   file=.../ECMWF/data/regular_ll/ea:0001:oper:20210702:1800:g/fc:ml.20240605.121554.hpcr5-vis5.6636793919176709.data, Offsets=[155833500, 234789140, 313744780, ]
GRIBJUMP_DEBUG Thread<PHONE_NUMBER>3520 starting
GRIBJUMP_DEBUG Thread<PHONE_NUMBER>7280 starting
GRIBJUMP_DEBUG Queued task 0
Queued task 0GRIBJUMP_DEBUG Waiting for 1 task...
GRIBJUMP_DEBUG task...
GRIBJUMP_DEBUG<PHONE_NUMBER>280 new job
GRIBJUMP_DEBUG Cache directory
GRIBJUMP_DEBUG Warning, cache persistence is disabled
GRIBJUMP_DEBUG Cache file /fc:ml.20240605.121554.hpcr5-vis5.6636793919176709.data.gribjump does not exist
GRIBJUMP_DEBUG Thread<PHONE_NUMBER>7280 exception: Read access outside data section: 2984368 + 32 > 2076480
GRIBJUMP_DEBUG All tasks complete
GRIBJUMP_DEBUGGRIBJUMP_DEBUG  Thread Thread<PHONE_NUMBER>7280 stopping (queue closed) data section: 2984368 +Thread Thread<PHONE_NUMBER>7280 stopping (queue closed) data section: 2984368 +
The offsets for step 1,2,3 seem right judging by grib dump-index on the FDB.

I'm using polytope (develop branch) and GribJump (develop branch,  either  may17 commit  or latest june5 give the same
result).

Debugging points to :  GribJumpDataAccessor.h (line 35)
throw std::runtime_error("Read access outside data section: " + std::to_string(offset) + " + " + std::to_string(size) + " > " + std::to_string(data_section_size))
Does this look familiar to you ?
Again, I realize this is work in progress. Ans I'm not implying this is a bug since I barely know what I'm doing when
creating the polytope request ! :-)

Full python script adaptation if ever you would consider running similar case.

cat 3D_shipping_route_era5_mabl_specific_humidity_jmb.py
import geopandas as gpd
import matplotlib.pyplot as plt
import pandas as pd
import pytest
from eccodes import codes_grib_find_nearest, codes_grib_new_from_file
from polytope.datacube.backends.fdb import FDBDatacube
from polytope.engine.hullslicer import HullSlicer
from polytope.polytope import Polytope, Request
from polytope.shapes import Ellipsoid, Path, Select
from tests.helper_functions import download_test_data
import logging
class TestReducedLatLonGrid:
def setup_method(self, method):
#JMB
logger = logging.getLogger(name)
logging.basicConfig(filename='lis_3D_shipping_route_era5_MABL_specific_humidity.log', encoding='utf-8', level=logging.DEBUG)
    self.options = {
        "config": [
            {"axis_name": "step", "transformations": [{"name": "type_change", "type": "int"}]},
            {"axis_name": "number", "transformations": [{"name": "type_change", "type": "int"}]},
            {
                "axis_name": "date",
                "transformations": [{"name": "merge", "other_axis": "time", "linkers": ["T", "00"]}],
            },
            {
                "axis_name": "values",
                "transformations": [
                    {"name": "mapper", "type": "reduced_ll", "resolution": 1441, "axes": ["latitude", "longitude"]}
                ],
            },

#JMB                {"axis_name": "latitude", "transformations": [{"name": "reverse", "is_reverse": True}]},
{"axis_name": "longitude", "transformations": [{"name": "cyclic", "range": [0, 360]}]},
]
}
#ea:0001:oper:20210701:1800:g
self.config = {"class": "od", "stream": "wave"}
    self.config = {"class": "ea", "stream" : "oper" }

    self.fdbdatacube = FDBDatacube(self.config, axis_options=self.options)
    self.slicer = HullSlicer()
    self.API = Polytope(datacube=self.fdbdatacube, engine=self.slicer, axis_options=self.options)

def find_nearest_latlon(self, grib_file, target_lat, target_lon):
    messages = grib_file

    # Find the nearest grid points
    nearest_points = []
    for message in [messages[0]]:
        nearest_index = codes_grib_find_nearest(message, target_lat, target_lon)
        nearest_points.append(nearest_index)

    return nearest_points

@pytest.mark.internet
@pytest.mark.skip(reason="can't install fdb branch on CI")
def test_reduced_ll_grid(self):
    shapefile = gpd.read_file(".../ECMWF/EARTHKIT_POLYTOPE/examples/data/Shipping-Lanes-v1.shp")
    geometry_multiline = shapefile.iloc[2]
    geometry_object = geometry_multiline["geometry"]

    lines = []
    i = 0

    #JMB

print("geometry_object=",geometry_object)
print("len(geometry_object,geoms)=",len(geometry_object.geoms))
#JMB        for line in geometry_object[:7]:
for line in list(geometry_object.geoms)[:7]:
for point in line.coords:
point_list = list(point)
if list(point)[0] < 0:
point_list[0] = list(point)[0] + 360
lines.append(point_list)
    # Append for each point a corresponding step

    new_points = []
    for point in lines[:7]:
        new_points.append([point[1], point[0], 1])

    print("new_points=",new_points)

    # Pad the shipping route with an initial shape

    padded_point_upper = [0.24, 0.24, 1]
    padded_point_lower = [-0.24, -0.24, 1]

padded_point_upper = [5.0, 5.0, 1]
padded_point_lower = [-5.0, -5.0, 0]
    initial_shape = Ellipsoid(["latitude", "longitude", "step"], padded_point_lower, padded_point_upper)

    # Then somehow make this list of points into just a sequence of points

    ship_route_polytope = Path(["latitude", "longitude", "step"], initial_shape, *new_points)
    print("JMB ship_route_polytope=",ship_route_polytope)

    request = Request(
        ship_route_polytope,
        Select("date", [pd.Timestamp("20210702T180000")]),
        Select("domain", ["g"]),
        Select("expver", ["0001"]),
        Select("param", ["133"]),
        Select("levelist", ["137"]),
        Select("class", ["ea"]),
        Select("stream", ["oper"]),
        Select("levtype", ["ml"]),
        Select("type", ["fc"]),
    )
    print("JMB request=",request)

    result = self.API.retrieve(request)
    result.pprint()

    print("JMB: after request result=",result)
    lats = []
    lons = []
    eccodes_lats = []
    eccodes_lons = []
    tol = 1e-8

#JMB        f = open("./tests/data/wave.grib", "rb")
f = open(".../ECMWF/data/regular_ll/202107/era5_202107_regular_ll.grib", "rb")
messages = []
print("JMB before codes_grib_new_from_file ",flush=True)
message = codes_grib_new_from_file(f)
print("JMB after codes_grib_new_from_file message=",message,flush=True)
messages.append(message)
    leaves = result.leaves
    for i in range(len(leaves)):
        #JMB
        print("JMB i, leaves=",i,leaves)
        cubepath = leaves[i].flatten()
        lat = cubepath["latitude"]
        lon = cubepath["longitude"]
        del cubepath
        lats.append(lat)
        lons.append(lon)
        nearest_points = codes_grib_find_nearest(message, lat, lon)[0]
        eccodes_lat = nearest_points.lat
        eccodes_lon = nearest_points.lon
        eccodes_lats.append(eccodes_lat)
        eccodes_lons.append(eccodes_lon)
        assert eccodes_lat - tol <= lat
        assert lat <= eccodes_lat + tol
        assert eccodes_lon - tol <= lon
        assert lon <= eccodes_lon + tol
        print(i)
    f.close()

    worldmap = gpd.read_file(gpd.datasets.get_path("naturalearth_lowres"))
    fig, ax = plt.subplots(figsize=(12, 6))
    worldmap.plot(color="darkgrey", ax=ax)

    plt.scatter(lons, lats, s=18, c="red", cmap="YlOrRd")
    plt.scatter(eccodes_lons, eccodes_lats, s=6, c="green")
    plt.colorbar(label="Temperature")
    plt.show()

a=TestReducedLatLonGrid()
a.setup_method(None)
a.test_reduced_ll_grid()

Cheers,
Jean-Marc

Hi Mathilde,
  Thanks for the help. I did mentionned I was not saying there was a bug ! :-)

End of listing with your suggestion:

JMB fdb.py assign_fdb_output_to_nodes: sorted_range_lengths= [2, 2, 2, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 2, 2, 2, 2, 1, 1, 1, 2, 2, 2, 2, 2, 1, 2, 1, 2, 2, 2, 2, 2, 1, 2, 1, 2, 2, 2, 2, 2, 1, 1, 1, 1, 2, 2, 3]
JMB fdb.py assign_fdb_output_to_nodes: len(sorted_fdb_range_nodes) 49
JMB fdb.py assign_fdb_output_to_nodes: len(sorted_range_lengths)= 49
JMB fdb.py assign_fdb_output_to_nodes: request_output_values= [[(array([0.01897763, 0.01898812]), array([3], dtype=uint64)), (array([0.0189309 , 0.01894377]), array([3], dtype=uint64)), (array([0.01861666, 0.01857756]), array([3], dtype=uint64)), (array([0.01925228, 0.01907585]), array([3], dtype=uint64)), (array([0.01757191]), array([1], dtype=uint64)), (array([0.01921032, 0.01924036]), array([3], dtype=uint64)), (array([0.01765965]), array([1], dtype=uint64)), (array([0.01865195, 0.01875971]), array([3], dtype=uint64)), (array([0.01804016]), array([1], dtype=uint64)), (array([0.01867436, 0.01855133]), array([3], dtype=uint64)), (array([0.01802109]), array([1], dtype=uint64)), (array([0.01856135, 0.01851414]), array([3], dtype=uint64)), (array([0.0183811]), array([1], dtype=uint64)), (array([0.01856325, 0.0186014 ]), array([3], dtype=uint64)), (array([0.01868962, 0.01883791]), array([3], dtype=uint64)), (array([0.019114  , 0.01894663]), array([3], dtype=uint64)), (array([0.01896332, 0.01910256]), array([3], dtype=uint64)), (array([0.01925371, 0.01914023]), array([3], dtype=uint64)), (array([0.01899813]), array([1], dtype=uint64)), (array([0.01924895]), array([1], dtype=uint64)), (array([0.0188484]), array([1], dtype=uint64)), (array([0.01918934, 0.01911066]), array([3], dtype=uint64)), (array([0.01867293, 0.01856039]), array([3], dtype=uint64)), (array([0.01952313, 0.01928614]), array([3], dtype=uint64)), (array([0.01834391, 0.01835249]), array([3], dtype=uint64)), (array([0.01955031, 0.01959131]), array([3], dtype=uint64)), (array([0.01830862]), array([1], dtype=uint64)), (array([0.01927422, 0.01930474]), array([3], dtype=uint64)), (array([0.01842688]), array([1], dtype=uint64)), (array([0.01934336, 0.01917456]), array([3], dtype=uint64)), (array([0.018414  , 0.01840876]), array([3], dtype=uint64)), (array([0.01935576, 0.01924084]), array([3], dtype=uint64)), (array([0.01839398, 0.01840304]), array([3], dtype=uint64)), (array([0.01930474, 0.01928519]), array([3], dtype=uint64)), (array([0.0183482]), array([1], dtype=uint64)), (array([0.01919602, 0.01917313]), array([3], dtype=uint64)), (array([0.01838063]), array([1], dtype=uint64)), (array([0.01911495, 0.01908301]), array([3], dtype=uint64)), (array([0.01833342, 0.01833294]), array([3], dtype=uint64)), (array([0.01902483, 0.01903055]), array([3], dtype=uint64)), (array([0.01832054, 0.01832245]), array([3], dtype=uint64)), (array([0.01890133, 0.01894282]), array([3], dtype=uint64)), (array([0.01829718]), array([1], dtype=uint64)), (array([0.01876972]), array([1], dtype=uint64)), (array([0.01830147]), array([1], dtype=uint64)), (array([0.018692]), array([1], dtype=uint64)), (array([0.01822947, 0.0182433 ]), array([3], dtype=uint64)), (array([0.01868342, 0.01871155]), array([3], dtype=uint64)), (array([0.0183544 , 0.01858614, 0.01869438]), array([7], dtype=uint64))]]
Traceback (most recent call last):
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 2022, in shape
result = a.shape
AttributeError: 'list' object has no attribute 'shape'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/jmb001/jmscript/PY/ECMWF/EARTHKIT_POLYTOPE/3D_shipping_route_era5_mabl_specific_humidity_mathilde.py", line 169, in
a.test_reduced_ll_grid()
File "/home/jmb001/jmscript/PY/ECMWF/EARTHKIT_POLYTOPE/3D_shipping_route_era5_mabl_specific_humidity_mathilde.py", line 117, in test_reduced_ll_grid
result = self.API.retrieve(request)
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/polytope/polytope.py", line 59, in retrieve
self.datacube.get(request_tree)
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/polytope/datacube/backends/fdb.py", line 75, in get
self.assign_fdb_output_to_nodes(output_values, fdb_requests_decoding_info)
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/polytope/datacube/backends/fdb.py", line 257, in assign_fdb_output_to_nodes
print("JMB fdb.py assign_fdb_output_to_nodes: np.shape(request_output_values)=",np.shape(request_output_values))
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 2024, in shape
result = asarray(a).shape
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 3 dimensions. The detected shape was (1, 49, 2) + inhomogeneous part.
GRIBJUMP_DEBUGGRIBJUMP_DEBUG Thread Thread<PHONE_NUMBER>7824 stopping (queue closed)22435554211392 stopping (queue closed)
Thread Thread<PHONE_NUMBER>7824 stopping (queue closed)22435554211392 stopping (queue closed)
It seems we got output values from GribJump now and they sure look like surface specific humidities around 18-19 g/Kg ... which is nice. There's a glitch remaining ...which appears minor ?
What is the meaning of  "resolution" :360  then ?   I thought we had to specify the data resolution (0.25 deg in this case). I guess this information is retrieved from FDB (GRIB) then.
I was also wondering about request about NEMO output. The software is not able to deal with such a grid yet (ORCA025), am I right ?
Anyway. Your help is much appreciated!
Jean-Marc
:-)

Hi @jmb64 ,
This is great! It looks like it's almost working :)
I believe maybe this print line in the code might be causing issues:
print("JMB fdb.py assign_fdb_output_to_nodes: np.shape(request_output_values)=",np.shape(request_output_values))
One of the problems of Polytope which happens here is that the output of Polytope isn't necessarily "box"-shaped so most of the numpy functions like shape() do not work on Polytope's output!
We've also just updated the develop branch of Polytope yesterday which might fix some of the last bugs!
For the resolution, it's currently maybe a bit counterintuitive. It was initially built around the octahedral grid, where the resolution was the number x in Ox, so somewhat related to the number of data points on a latitude line. We've tried to keep it somewhat "uniform" for the different grids and so the resolution for the regular lat/lon grid is the number of points on a latitude line (in this case it's 90/0.25=360). I hope this explains a bit more!
Unfortunately, I do not believe we support ORCA grids yet.
I hope this helps!

Mathilde,
I assure you ... this helps ! :-)

How can I reference the actual values (specific humidities) along the trajectory) in the script (duhhh) ?
I wrote them in fdb.py but ....
JM

Mathilde,
Finally,
Using leaves[i].result (?)  ...

JM


Due Dates not appeared in Projects in sir company

Duedates hain services me but Projects me nhi dikh rahin

need to find scenario


Incorrect timeout warnings when using AWS Lambda integration

sentry-sdk version: 0.17.8
AWS Lambda runtime: Python 3.8
My lambda function has a timeout of 29 seconds, but typically completes in <1s. When the lambda function is called multiple times in quick succession, the container for the lambda function is reused. When this happens, the threads that Sentry creates to manage the timeout warnings produce their warnings in subsequent function invocations (long after the function that they were created in has completed).

I think I have the same issue using sentry-sdk 0.18.0 and Python 3.7.9


Tuple literals not matching correctly

Summary
While trying to build a simple "event sourcing" style application I ran into problems with tuple matching.
Essentially if I build a tuple as a literal:
	let cc: Change = ((0,0), "bob", Dependency, (Link, "123"))

then the subsequent match matches incorrectly (not just a failed match but an incorrect match).
While, if I build the same tuple in two steps:
	let vv: DependencyOp = (Link, "123")
	let cc: Change = ((0,0), "bob", Dependency, vv)

The ponyc version is: 0.19.0-cf45938 [release]
Example Code
Bellow is a pared down version of my code that includes just the enough of the types in order to be able to trigger the problem. Note if Change has only the DependencyOp command in its type definition (versus including the NOP command) then the problem goes away and the tests succeed.
cat _test_tuples.pony
use "ponytest"
use "collections"

// -- types

type TimeStamp is (I64, I64)
type Principal is String

primitive NOP
primitive Dependency

primitive Link
primitive Unlink
primitive Spawn

type ID is String

type DependencyOp is ((Link, ID) | (Unlink, ID) | Spawn)

type Change is (
	//  when        principal   aspect       value
	  ( TimeStamp , Principal , NOP        , None           ) // <-- needed in order to trigger the bug
	| ( TimeStamp , Principal , Dependency , DependencyOp   )
	// ... other journal command types
)

// -- tests

actor Main is TestList

  new create(env: Env) =>
    PonyTest(env, this)

  new make() =>
    None

  fun tag tests(test: PonyTest) =>
		test(_TestTupleNestLiteral)
		test(_TestTupleNestConstructed)

// --

class iso _TestTupleNestLiteral is UnitTest
	fun name(): String => "tuplenest:literal"

	fun ref apply(h: TestHelper) =>
		// building the Change using the following literal tuple does _not_ matche correctly
		let cc: Change = ((0,0), "bob", Dependency, (Link, "123"))
		CheckChange.check(cc,h)

class iso _TestTupleNestConstructed is UnitTest
	fun name(): String => "tuplenest:constructed"

	fun ref apply(h: TestHelper) =>
		// building the Change using the following construction matches correctly
		let vv: DependencyOp = (Link, "123")
		let cc: Change = ((0,0), "bob", Dependency, vv)
		CheckChange.check(cc,h)

primitive CheckChange
	fun check(cc: Change, h: TestHelper) =>
		match(cc)
		| ( let t: TimeStamp , let p: Principal, let a: Dependency, let v: DependencyOp ) =>
			match (v)
			| (Link, let vw: ID)	=> h.complete(true)
			| (Unlink, let vw: ID)	=> h.fail("expected link - got unlink")
			| (Spawn)		=> h.fail("expected link - got spawn")
			end
		else
			h.fail("didn't match outer")
		end

Example Output
130 09:57 stewart@flatfoot:~/opt/pony/bugs/tuple-literal$ export PONYC=~/opt/pony/bleed/build/release/ponyc; ${PONYC} -d; ./tuple-literal ; ${PONYC} -v
Building builtin -> /home/stewart/opt/pony/bleed/packages/builtin
Building . -> /home/stewart/opt/pony/bugs/tuple-literal
Building ponytest -> /home/stewart/opt/pony/bleed/packages/ponytest
Building time -> /home/stewart/opt/pony/bleed/packages/time
Building collections -> /home/stewart/opt/pony/bleed/packages/collections
Generating
 Reachability
 Selector painting
 Data prototypes
 Data types
 Function prototypes
 Functions
 Descriptors
Writing ./tuple-literal.o
Linking ./tuple-literal
Warning: environment variable $CC undefined, using cc as the linker
1 test started, 0 complete: tuplenest:literal started
2 tests started, 0 complete: tuplenest:constructed started
2 tests started, 1 complete: tuplenest:literal complete
2 tests started, 2 complete: tuplenest:constructed complete
**** FAILED: tuplenest:literal
expected link - got spawn
---- Passed: tuplenest:constructed
----
---- 2 tests ran.
---- Passed: 1
**** FAILED: 1 test, listed below:
**** FAILED: tuplenest:literal
0.19.0-cf45938 [release]
compiled with: llvm 3.9.1 -- cc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

At first glance, I think this might be a "tuple of unions vs union of tuples" issue, which is a known limitation of the compiler. But I could be wrong, as I haven't looked at it in detail.
It's also possible this is related to the change in #1937. This could be confirmed by testing against ponyc version 0.14.0 or earlier.

I tested this with the Wallaroo Labs ponyc version (which is ponyc 0.9 more or less) and it worked then. can confirm it doesnt work with 0.19.0

Okay, sounds like it's a side effect of #1937, then.

@jemc do you know if you ever looked into this? i'm assuming no.

No, I haven't looked into it.


Docs?

Hi,
Is there any documentation for your plugin? I don't have access to configure my server with the dependencies requires to run the Tests so I can't see any example of how to use this.
Thanks in Advance,
Bob.

Hey there.
I don't currently have any docs, but give a couple of days and I can get them together.
Thanks for the interest.

I managed to get it working, but since I didn't install Cake with composer I was unable to install this plugin with composer neither. I had to manually app::import the Vendor file:
App::import('Vendor', 'firebase/php-jwt/src/JWT');
Then I had to comment out the namespace line in JWT.php:
namespace Firebase\JWT;
This enabled me to call the code and decode functions which all works nicely.
Thanks!

I had no trouble installing through composer, just put
{ "require": { "t73biz/cakephp2-jwt-auth": "1.0.6" } } into composer.json under /app
However, I'd really like an example of how to use this plugin. Surely someone has some code lying around that you could upload after taking out the sensitive bits? Even if you post basic usage right in this comment section.

I'd recommend against using this library for now, clearly whoever's in charge is not interested. There's a valuable pull request that has some documentation if you're bent on using this library. The request is being completely ignored by the owner of this project. :}
Also, it does not install the latest update to firebase's JWT.php when composing. This version includes an exploit involving empty keys.
To get it working for myself, I did the following:

installed JWT.php with: composer require firebase/php-jwt
commented out the namespace as spacebiscuit said (THANK YOU!)
opened JWT.php and in decode, I turned all exceptions into die() because Cake was getting fussy

And I followed this tutorial.
https://www.sitepoint.com/php-authorization-jwt-json-web-tokens/
I cannot say with confidence that I've done the best job, but JWTs are working for me without having to use a Cake plugin. My requirements are basic and may not mirror your company's needs.

I do appologize for the delay. I will be adding in the PR https://github.com/t73biz/cakephp2-jwt-auth/pull/9 to address the issues for documentation.


java.lang.IncompatibleClassChangeError on Android 5.0-6.0

Exception message in runtime:
Class 'ua.pl.mik.app.DaggerAppComponent$SomeLongNameBuilder' does not implement interface 'ua.pl.mik.app.net.ApiComponent$Builder' in call to 'ua.pl.mik.app.net.ApiComponent ua.pl.mik.app.net.ApiComponent$Builder.build()
Class names was changed a bit.
On Android 7.0+ same apks (main + androidTest) working fine.

If you have a custom AndroidJUnitRunner, please make sure that the class literal of your Application class is not used on AndroidJUnitRunner#newApplication.
It would be helpful if you can share a minimal project that can reproduce this issue.

Don't call TestApp.class.getName() on newApplication().
Instead, use a string literal of the Application class name.
return super.newApplication(cl, "your.app.test.TestApp", context);

Tried using raw string: still same exception.
Currently can't identify what of my code cause this to setup minimal project... The only thing I see that: this classes was generated by Dagger (subcomponent builders).

Also I use multidex for debug builds, but as I see, all these classes in exception located in a single dex...

Just found the corresponding issue in the google bug tracker, if anyone is interested.
https://issuetracker.google.com/issues/75295983


Allow specifying msvc manifest file to use

Hello, I'd like to integrate this tool in my nix-based msvc toolchain, replacing vsdownload.py from msvc-wine. But in order to do this, I need to ensure I can reproducibly download the source files from Microsoft.
To do so, vsdownload.py provides the ability to specify a manifest file, which is then used as-is instead of fetching the file from microsoft. See this. Downloading files from this pinned manifest is fully reproducible, as it pins the hash of all its sub-resources.
Having a similar ability in xwin would be great.

Makes sense!

Yes please! This would be very nice for CI builds as well.

So as it turns out, there's still a reproducibility issue somewhere. On one machine, I get result/sdk/include/shared/ndis/{status.h,version.h}, and on the other machine the same files end up in result/sdk/include/um/alljoyn_c instead. That sounds like a threading/rayon issue, and I'm currently trying to rebuild with RAYON_NUM_THREADS=1.

Actually, it looks like it might be this: https://github.com/Jake-Shadle/xwin/blob/main/src/splat.rs#L317

Removing that check seems to make the output reproducible, or at least a lot more reproducible: https://github.com/K900/xwin/commit/b1de3341b3579df2334849bc9aaf4246a2c73451

Ahh ok, that's unfortunate, if every SDK header is written then the output will contain a lot of duplicates, so I'll need to change this to filter out those duplicates in a deterministic manner.

I tried switching it to use a BTreeMap and that seems to work, but only in single thread.


Cannot parse eg/maildir-as-a-sample/new/arf-cannot-detect-recipient-address.eml

63835ce4a57be5784118183f673fb118f4352eb0
The email has no recipient address.

Fixed at commit be6d9795c1d74615896c7dbe56dce1eff300df1f in Pull Request #37


DOC: edited t_span parameter description in integrate.solve_ivp

Reference issue
Closes issue #16978
What does this implement/fix?
A simple documentation fix; as pointed out, t_span is an array_like object but was described as a tuple.
Additional information

t_span is given as an argument to map: t0, tf = map(float, t_span). Therefore, it just needs to be a 2-member iterable, which means that any sequence object would in principle work as long as its members can be interpreted by float.

t_span is given as an argument to map: t0, tf = map(float, t_span). Therefore, it just needs to be a 2-member iterable, which means that any sequence object would in principle work as long as its members can be interpreted by float.

@Patol75 Hello, thank you for your feedback!
I tried to edit the parameter's type and description based on the information you provided. Since I didn't know how to express "its members can be interpreted by the float function" as a type, I wrote it down within the description paragraph.
The changes have been commited. If you think there is a better way to express it then feel free to tell me :)

Thanks @korneix.


Nominatim as fallback?

Is there any plan to also support Nominatim (http://wiki.openstreetmap.org/wiki/Nominatim), from openstreetmap, as a fallback geocoding service rather than google maps?
I think it would be more convenient as Nominatim do not limit in the number of daily requests...
EDIT: geocoding service from mapquest used to be free without limit too but I just saw it is no longuer the case :( . But maybe there is some other completely free geocoding service that could be used as fallback... Just found this one: http://services.gisgraphy.com/public/geocoding.html but it requiere you to host your own server to use it for free (else you have to pay if you want to use premium servers)

I have yet to find a better geocoding service than google. and anyway it is only used as a fallback so you are very unlikely to hit the quota limit ;)

Is there any plan to also support Nominatim (http://wiki.openstreetmap.org/wiki/Nominatim), from openstreetmap, as a fallback geocoding service rather than google maps?
I think it would be more convenient as Nominatim do not limit in the number of daily requests...
EDIT: geocoding service from mapquest used to be free without limit too but I just saw it is no longuer the case :( . But maybe there is some other completely free geocoding service that could be used as fallback... Just found this one: http://services.gisgraphy.com/public/geocoding.html but it requiere you to host your own server to use it for free (else you have to pay if you want to use premium servers)

hello im having a trouble accesing nominatim using fetch api its said i need to have a valid user agent or http referrer


Support for working directory

Hello,
Thanks for awesome action. Is there any plan to add support for a working directory? Currently, there is no way to use the action for mono repo because go codebase resides in subdirectory.

An optional working-directory param has been added https://github.com/n8maninger/action-golang-test/releases/tag/v2.1.0
jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: '1.21'
      - uses: n8maninger/action-golang-test@v2
        with:
          working-directory: ./sub-directory

@n8maninger thanks for that feature. I tested it and looks like annotations do not work :(

In my case the go.mod is placed in the backend folder.

Looks like I missed a spot for the working directory. I've fixed it and added a test case for the future.

@n8maninger It works now. Awesome! Thanks for that feature.


null is not an object (evaluating 'child.props')

Bug
When trying to conditionally render a SVG, if the condition is false (and therefore not rendered) using
      {showMessageButton && (
        <MessageIcon svgClassName="w-4 h-4 text-gray-600" />
      )}

The following crash happens.
MessageIcon.tsx:
import React from "react";
import { View } from "react-native";
import Svg, { Path } from "react-native-svg";

const MessageIcon = ({ svgClassName }) => {
  return (
    <View>
      <Svg className={svgClassName} viewBox="0 0 24 24">
        <Path
          fill="currentColor"
          d="M9.98999 22.78C9.38999 22.78 8.81999 22.48 8.42999 21.95L7.23001 20.35C7.23001 20.36 7.18 20.33 7.16 20.33H6.79001C3.37001 20.33 1.25 19.4 1.25 14.79V10.79C1.25 6.58001 3.82001 5.48001 5.98001 5.29001C6.22001 5.26001 6.50001 5.25 6.79001 5.25H13.19C16.81 5.25 18.73 7.17001 18.73 10.79V14.79C18.73 15.08 18.72 15.36 18.68 15.63C18.5 17.76 17.4 20.33 13.19 20.33H12.79L11.55 21.95C11.16 22.48 10.59 22.78 9.98999 22.78ZM6.79001 6.75C6.56001 6.75 6.34 6.76 6.13 6.78C3.81 6.98 2.75 8.25001 2.75 10.79V14.79C2.75 18.22 3.81001 18.83 6.79001 18.83H7.19C7.64 18.83 8.14999 19.08 8.42999 19.44L9.63 21.05C9.85001 21.35 10.13 21.35 10.35 21.05L11.55 19.45C11.84 19.06 12.3 18.83 12.79 18.83H13.19C15.73 18.83 17 17.76 17.19 15.48C17.22 15.24 17.23 15.02 17.23 14.79V10.79C17.23 8.00001 15.98 6.75 13.19 6.75H6.79001Z"
        />
        <Path
          fill="currentColor"
          d="M9.99023 14.1899C9.43023 14.1899 8.99023 13.7399 8.99023 13.1899C8.99023 12.6399 9.44023 12.1899 9.99023 12.1899C10.5402 12.1899 10.9902 12.6399 10.9902 13.1899C10.9902 13.7399 10.5502 14.1899 9.99023 14.1899Z"
        />
        <Path
          fill="currentColor"
          d="M13.1899 14.1899C12.6299 14.1899 12.1899 13.7399 12.1899 13.1899C12.1899 12.6399 12.6399 12.1899 13.1899 12.1899C13.7399 12.1899 14.1899 12.6399 14.1899 13.1899C14.1899 13.7399 13.7399 14.1899 13.1899 14.1899Z"
        />
        <Path
          fill="currentColor"
          d="M6.7998 14.1899C6.2398 14.1899 5.7998 13.7399 5.7998 13.1899C5.7998 12.6399 6.2498 12.1899 6.7998 12.1899C7.3498 12.1899 7.7998 12.6399 7.7998 13.1899C7.7998 13.7399 7.3498 14.1899 6.7998 14.1899Z"
        />
        <Path
          fill="currentColor"
          d="M17.9401 16.29C17.7401 16.29 17.5401 16.21 17.4001 16.06C17.2401 15.9 17.1701 15.67 17.2001 15.45C17.2301 15.24 17.2401 15.02 17.2401 14.79V10.79C17.2401 8.00001 15.9901 6.75 13.2001 6.75H6.80012C6.57012 6.75 6.35015 6.76 6.14015 6.78C5.92015 6.81 5.69013 6.72999 5.53013 6.57999C5.37013 6.41999 5.28012 6.20001 5.30012 5.98001C5.48012 3.82001 6.59012 1.25 10.8001 1.25H17.2001C20.8201 1.25 22.7401 3.17001 22.7401 6.79001V10.79C22.7401 15 20.1701 16.1 18.0101 16.29C17.9801 16.29 17.9601 16.29 17.9401 16.29ZM6.92015 5.25H13.1901C16.8101 5.25 18.7301 7.17001 18.7301 10.79V14.66C20.4301 14.24 21.2301 12.99 21.2301 10.79V6.79001C21.2301 4.00001 19.9801 2.75 17.1901 2.75H10.7901C8.59014 2.75 7.35015 3.55 6.92015 5.25Z"
        />
      </Svg>
    </View>
  );
};

export default MessageIcon;

TypeError: null is not an object (evaluating 'child.props')

This error is located at:
    in TailwindCssReactNative.NoName
    in Unknown (created by ListingSeller)
    in ListingSeller (created by Listing)
    in RCTScrollContentView (created by ScrollView)
    in RCTScrollView (created by ScrollView)
    in ScrollView (created by ScrollView)
    in ScrollView
    in Unknown
    in Unknown
    in TailwindCssReactNative.NoName
    in Unknown (created by Listing)
    in RCTSafeAreaView
    in SafeAreaView
    in TailwindCssReactNative.NoName
    in Unknown (created by Listing)
    in Listing (created by SceneView)
    in StaticContainer
    in EnsureSingleNavigator (created by SceneView)
    in SceneView (created by BottomTabView)
    in RCTView (created by View)
    in View (created by Screen)
    in RCTView (created by View)
    in View (created by Background)
    in Background (created by Screen)
    in Screen (created by BottomTabView)
    in RNSScreen (created by AnimatedComponent)
    in AnimatedComponent
    in AnimatedComponentWrapper (created by Screen)
    in MaybeFreeze (created by Screen)
    in Screen (created by MaybeScreen)
    in MaybeScreen (created by BottomTabView)
    in RNSScreenNavigationContainer (created by ScreenContainer)
    in ScreenContainer (created by MaybeScreenContainer)
    in MaybeScreenContainer (created by BottomTabView)
    in RCTView (created by View)
    in View (created by SafeAreaInsetsContext)
    in SafeAreaProviderCompat (created by BottomTabView)
    in BottomTabView (created by BottomTabNavigator)
    in Unknown (created by BottomTabNavigator)
    in BottomTabNavigator (created by BottomNavigation)
    in BottomNavigation (created by App)
    in DeviceMediaProvider (created by TailwindProvider)
    in ColorSchemeProvider (created by TailwindProvider)
    in PlatformProvider (created by TailwindProvider)
    in TailwindProvider (created by App)
    in EnsureSingleNavigator
    in BaseNavigationContainer
    in ThemeProvider
    in NavigationContainerInner (created by App)
    in $29383e587d62412a$export$9f8ac96af4b1b2ae (created by NativeBaseProvider)
    in ToastProvider (created by NativeBaseProvider)
    in PortalProvider (created by NativeBaseProvider)
    in HybridProvider (created by NativeBaseProvider)
    in ResponsiveQueryProvider (created by NativeBaseProvider)
    in RNCSafeAreaProvider (created by SafeAreaProvider)
    in SafeAreaProvider (created by NativeBaseProvider)
    in NativeBaseConfigProviderProvider (created by NativeBaseProvider)
    in NativeBaseProvider (created by App)
    in AuthProvider (created by App)
    in App (created by ExpoRoot)
    in ExpoRoot
    in RCTView (created by View)
    in View (created by AppContainer)
    in DevAppContainer (created by AppContainer)
    in RCTView (created by View)
    in View (created by AppContainer)
    in AppContainer
    in main(RootComponent)
at node_modules/react-native/Libraries/Core/ExceptionsManager.js:95:4 in reportException
at node_modules/react-native/Libraries/Core/ExceptionsManager.js:141:19 in handleException
at node_modules/react-native/Libraries/Core/setUpErrorHandling.js:24:6 in handleError
at node_modules/expo-error-recovery/build/ErrorRecovery.fx.js:12:21 in ErrorUtils.setGlobalHandler$argument_0
info Opening /Users/lewis/Desktop/hoardboard/node_modules/tailwindcss-react-native/dist/with-styled-children.js with code
error spawn code ENOENT

Library version: 12.3.9

I am having the same issue. Any solutions found?

i am having the same problem

I'm having the same issue, any solution?

I'm having the same problem
Any solution?

I checked the reproduction on the newest main and couldn't spot the problem. I will close this issue then. Feel free to comment here if something is wrong and we can always reopen it then.

It's because of the curly braces at line 4:
const MessageIcon = ({ svgClassName })
should be
const MessageIcon = ( svgClassName )
Think about it that way, becuase you implemented it the same way as implementing props:
<User isOnline={true}/>
you get it the same way as props:
const User = (props)=>{}
if you implement it the same way as route for example in react navigation:
navigation.navigate("UserScreen", {shownUser: user}
you add the curly braces becuase you had them before ^
const UserScreen = ({route})=>{}


Investigate intermittent ElasticSearch unresponsiveness

Occasionally, every node in the ElasticSearch production cluster for www.bloomapi.com becomes unresponsive, resulting in the site going down entirely. Investigate and fix this.

Downtime has been investigated. Root cause identified to be OutOfMemory exceptions on the production ElasticSearch cluster. Increased size of memory for machines in the cluster to resolve the issue.


Angular2 and Roadmap

Hi,
In #1096 @ProLoser mentioned the complexity that the project had reached and he suggested some interesting points. Even before in #884 quite some concernes were raised about the status of the project, in fact @brianfeister said that it was starting to be quite difficult to merge old and new PRs due to the complexity of the project. These two issues have been there for a year or more.
In the meanwhile the Roadmap was lastly updated on Nov 26, 2014.
For those of us already looking forward to the Angular2 switch would be very interesting to know if there is any short/long planning or an open discussion going on about the future of this project, especially if you could see an Angular2 version of it or not.
As I've understood to keep on improving this version for angular1 many things should me modularized and complexity broken down, and for angular2 I guess it seems to be a completely re-write almost from zero if I'm not wrong.
Some project as https://github.com/OptimalBPM/angular-schema-form-dynamic-select are based on ui-select and before looking for an A2 re-write it seems a good idea to check the status and the roadmap of this project.
Thanks

There isn't any planning/discussion/roadmap for an angular2 version going on as far as I know.
Current efforts are focused on resolving issues with the library...

See also: #784

don't know about the roadmap but the amount of unmerged pull requests is obnoxious. Even solution of the bugs are not merged, who is in the team of the project anyway?

Who would like to be? I see a lot of people complain, but I don't see many people stepping up and really taking ownership of the project as a whole


Fix 603 2010 upgrade ( don't merge )

Upgrading vstats was broken when going from version 6.0.3 to version 20.10.R1. Looking at the Jenkins log, we saw the last successful upgrade of this type was January 28. http://<IP_ADDRESS>:8080/job/Scheduled-Tests/job/UPGRADE-GCP-SA-6.0.3-20.10.R1/75/
There were a number of changes in early February that seem to have broken it, specifically changing this:

name: Set variable for upgrade versions
set_fact:
upgrade_60_2010R1:
"{{ upgrade_from_version|upper|replace('R','') is search('6.0.') and upgrade_to_version|upper|replace('R','') is search('20.10.1') }}"

to this

name: Set variable for upgrade versions
set_fact:
upgrade_60_2010R1:
"{{ upgrade_from_version|upper|replace('R','') is search('6.0.') and upgrade_to_version|upper|replace('R','') is version('20.10.5', '>=') }}"

It looks like the scripts that do not work for 20.10.R1 do not work for anything >= 20.10.R1.
My version is:

name: Set variable for upgrade versions
set_fact:
upgrade_60_2010R1:
"{{ upgrade_from_version|upper|replace('R','') is search('6.0.') and upgrade_to_version|upper|replace('R','') is search('20.10.') }}"

If we only want upgrade_60_2010R1 to be true for certain versions, we will need to refine the check. If we want upgrade_60_2010R1 to be true for all versions >= 20.10.R1, I suggest renaming the variable to upgrade_60_2010, so that the code is clear.
Passing test: http://<IP_ADDRESS>:8080/job/nesc/job/UPGRADE-GCP-SA/

duplicate code


python310Packages.google-cloud-error-reporting: 1.8.1 -> 1.8.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python310Packages.google-cloud-error-reporting is: Stackdriver Error Reporting API client library
meta.homepage for python310Packages.google-cloud-error-reporting is: https://github.com/googleapis/python-error-reporting
meta.changelog for python310Packages.google-cloud-error-reporting is: https://github.com/googleapis/python-error-reporting/blob/v1.8.2/CHANGELOG.md
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.8.2 with grep in /nix/store/3j2d0psiksf13prwnywkllccrywv7pbp-python3.10-google-cloud-error-reporting-1.8.2
found 1.8.2 in filename of file in /nix/store/3j2d0psiksf13prwnywkllccrywv7pbp-python3.10-google-cloud-error-reporting-1.8.2

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath
python310Packages.google-cloud-error-reporting
python311Packages.google-cloud-error-reporting

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/3j2d0psiksf13prwnywkllccrywv7pbp-python3.10-google-cloud-error-reporting-1.8.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python310Packages.google-cloud-error-reporting https://github.com/r-ryantm/nixpkgs/archive/3e11586e50a8349de1c95b792359fffa4c098fd4.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/3j2d0psiksf13prwnywkllccrywv7pbp-python3.10-google-cloud-error-reporting-1.8.2
ls -la /nix/store/3j2d0psiksf13prwnywkllccrywv7pbp-python3.10-google-cloud-error-reporting-1.8.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

python310Packages.google-cloud-error-reporting
python311Packages.google-cloud-error-reporting

Maintainer pings
cc @SuperSandro2000 for testing.

@ofborg build python310Packages.google-cloud-error-reporting python311Packages.google-cloud-error-reporting

@ofborg eval

Result of nixpkgs-review pr 215373 run on aarch64-darwin 1

2 packages built:

python310Packages.google-cloud-error-reporting
python311Packages.google-cloud-error-reporting


Improve React Native support

Usage example in app
This is slightly breaking, but I'm only breaking it because the previous implementation simply didn't work, to import CoveoAnalytics inside react native you would need to do the following:
import {CoveoAnalyticsClient} from 'coveo.analytics/dist/library.es';

With this PR there is now a new sub package where you can import everything you need:
import {
  ReactNativeRuntime,
  CoveoAnalyticsClient,
  ...
} from 'coveo.analytics/react-native';

I also rearranged the options so they make more sense, before you would need to implement your own "visitorIdProvider", build your own "baseUrl" from the endpoint & versions. It's not something that's straightforward or dev friendly.
I removed the hard dependency "@react-native-async-storage/async-storage" for multiple reason:

It's a package 99.9% of the lib's users won't need.
It came with a lot of vulnerabilities.
It's support inside an OOTB React Native project is terrible, you get an error because of it OOTB (see image) that requires additional work to be fixed (Especially with expo where you need to "eject" the app to use it properly).
There are alternative libraries, we shouldn't force the use of this one.
Implementing a custom Storage is pretty easy.

This should probably be a minor version bump instead of a patch bump.

It is "version": "2.22.0",


Update GitHub org owners to match reality

GitHub org ownership currently seems to reflect a mishmash of historical state of affairs.
At the time of writing, the org owners are

@domenkozar
@garbas
@zimbatm
@edolstra
@lassulus (added as interim manager)

This is also documented in the recently created org repo, but without any plan of how to proceed.
There are no provisions in the constitution for how to deal with ownership, and what's there leaves open to interpretation if the steering committee should also have org owners status.
Whatever the outcome of the 2024 elections, I would ask for actual org owners to be responsible and available, not just exercise silent authority.
@Lassulus: Feel free to move this to https://github.com/NixOS/org if that's a better place to handle the issue.

The new SC should ultimately handle the org ownership. Of course it's also possible for them to delegate it to a dedicated team. So I would wait for the new SC to decide what they want to do with this responsibility and update the org owners afterwards? Or is there a reason to do this before the first SC is established?

@fricklerhandwerk was there any particular event that prompted you to write this message? AFAIK we are already operating as interim managers and not official "owners". This could be clarified here: https://github.com/NixOS/org/blob/main/doc/github.md#github
If you need something specific to get done, please reach out to https://matrix.to/#/#org_owners:nixos.org or by creating an issue on this repo.

@zimbatm nothing specific. But these things communicate something: who's responsible and in charge, and what to expect. If it's not aligned with what is actually the case, we'll be perpetuating the opaqueness and feeling of "something's weird" that's been plaguing the ecosystem for many years.
For example, for people not in the very inner circle it's not clear who will actually react to requests, how long it will take, and whether the resulting action will be generally accepted.

Fixed! See https://github.com/NixOS/org/blob/main/doc/github-org-owners.md


Fix client publisher metric

We need to increment the failed metric only if we get an error.
Peer reviewed by @boy-uber

Coverage remained the same at 80.903% when pulling e8354b4082b97240f5dec720a29763ec0cb1adc9 on fix_metric into fd3d0a1958e793bb78438c3f60ab6b4193157a02 on master.


AoT border renders on top of taskbar on Win11 when maximized

Microsoft PowerToys version
0.53.1
Running as admin

[X] Yes

Area(s) with issue?
Always on Top
Steps to reproduce

Make a windows always on top
Maximize it

If you cahnge to another window it disappears again. It doesn't happen on wvery app the same way. In opera GX for example the boreder appears and goes away while typing.
✔ Expected Behavior
No response
❌ Actual Behavior
No response
Other Software
No response

For me on maximized window only the bottom part of the border (between window and taskbar) is visible.

Here is another example of this funny bug
PowerToys - Always on Top:

WindowTop:

Not only that the border idea copied from my app, you also did super bad job!
Keep it up

/dup https://github.com/microsoft/PowerToys/issues/19860