ntvs 2015, debugging apps with large objects causes 100% cpu in node.exe

I have a pretty complex node app,  debugging in 2013 is fast (mostly) but with 2015 performance tanks when my objects are massive.
the problem is when setting a breakpoint and stepping through my code.  every step or variable-hoverover causes node.exe to max out the cpu, for 15 sec or so usually  (sometimes, especially with big buffers it just doesn't recover at all).  It looks to have to do with some debugger-to-node roundtripping when there is a lot of memory used (when my user-defined objects are very large, multi-megabyte).   If I run the same code-path but with smallish objects the performance is acceptable.
after seeing the issue, I upgraded to the latest ntvs, typescript, and node.exe versions (as of today) and still having the issue on VS 2015.
The performance in vs2013 (update 5) is fine.

Closing, as this appears to be a duplicate of #387, but please let us know if you believe otherwise, and we'll re-open.

this is certainly related, though I mostly suffered when stepping through with the debugger, which is not described in the #387 ticket.    up to you if you feel they are similar enough issues to fold together!

Ahh, yes - you're right. It also appears that your issue is more specific to VS 2015.

And have you tried different versions of node to see if that has any impact?

Re the statement when my objects are massive - so I assume if you don't have massive objects performance is fine? Do you happen to have one of these massive objects displayed in the Watch / Locals window? Or a conditional breakpoint that uses an expression which touches one of these massive objects?

@mousetraps Yes, you are right, I only have problems with VS2015.  I tried other versions of node (0.12.x x86 , and 4.2.1 x86 + x64)  and they all had identical issues.
@billti correct,  small objects have much better performance.  This is with nothing in watch/locals, and no conditional breakpoints.
I can't provide a functional example, but here is the section of code where I reproed the issue:

					if (contentFilePath == null) {
						//empty promise
						var resolver = Promise.defer<Buffer>();
						contentPromise = resolver.promise;
						resolver.resolve(null);
					} else {
						contentPromise = nlib.fs.readFileAsync(contentFilePath);
					}

					Promise.join(userResponseFilePromise, contentPromise, (userResponseBuffer, contentBuffer) => {

						//construct final user-response object (including content)
						var userResponse = <ioDatatypes.IUserResponse>JSON.parse(userResponseBuffer.toString());

						//replaceNodes(userResponse.pageResponses[0], userResponse.pageResponses[0].pageRequest.suppressJson);

						var totalSize = 0;
						if (userResponseBuffer != null) {
							totalSize += userResponseBuffer.length
						}
						if (contentBuffer != null) {
							totalSize += contentBuffer.length
						}
						var data: string;
						var name: string;
						var encoding: string;
						var size: number;
						var isError = false;
						var statusCode = 200;
						if (contentBuffer != null) {

							switch (contentFileName) {

								case "content.jpeg":
									data = contentBuffer.toString("base64");
									name = contentFileName;
									encoding = "base64";
									size = data.length;
									break;
								case "content.png":
									data = contentBuffer.toString("base64");
									name = contentFileName;
									encoding = "base64";
									size = data.length;
									break;
								case "content.pdf":
									data = contentBuffer.toString("base64");
									name = contentFileName;
									encoding = "base64";
									size = data.length;
									break;
								case "content.html":
									data = contentBuffer.toString("utf8");
									name = contentFileName;
									encoding = "utf8";
									size = data.length;
									break;
								case "content.text":
									data = contentBuffer.toString("utf8");
									name = contentFileName;
									encoding = "utf8";
									size = data.length;
									break;
								case "content.json":
									data = contentBuffer.toString("utf8");
									name = contentFileName;
									encoding = "utf8";
									size = data.length;
									break;
								default:
									log.error("unknown content file name", contentFileName);
									isError = true;
							}
						} else {
							log.error("contentBuffer is null");
							isError = true;
						}

IIRC when the contentBuffer gets large,(like  15MB), the VS2015 ide is very slow on stepping through the (userResponseBuffer, contentBuffer) => {} method when debugging, with each step (F10) causing node.exe to max out a CPU for a long time.
Hope that helps.

btw, in case this may be significant for reproducability,  my CPU is a i5, 4 core 3.6ghz,  and I have 32gb ram, and my os+programs+code is on a SSD,    my os is win10

Looks like this is a duplicate of #387. Closing, but let us know if you think otherwise.


Always getting S2I failed to install [assemble run]

Every time when I try to build, I get the following error and I'm not sure why, because the files are available in the image and are executable. Am I missing something ? What can be the reasons for this message ?
Log of ' s2i --loglevel 5 --ref master build https://github.com/cchet-thesis-msc/app-camel cchet/openshift-app-camel:0.1'
cchet/openshift-app-camel:0.1
I0925 12:04:26.534099    1559 main.go:69] Running S2I version "v1.1.7"
I0925 12:04:26.534186    1559 util.go:57] Getting docker credentials for cchet/openshift-app-camel:0.1
I0925 12:04:26.537468    1559 docker.go:503] Using locally available image "cchet/openshift-app-camel:0.1"
I0925 12:04:26.539475    1559 main.go:159]
Builder Name:			App Camel JEE
Builder Image:			cchet/openshift-app-camel:0.1
Source:				https://github.com/cchet-thesis-msc/app-camel
Source Ref:			master
Output Image Tag:
Environment:
Labels:
Incremental Build:		disabled
Remove Old Build:		disabled
Builder Pull Policy:		if-not-present
Previous Image Pull Policy:	if-not-present
Quiet:				disabled
Layered Build:			disabled
Docker Endpoint:		unix:///var/run/docker.sock
Docker Pull Config:		/home/herzog/.docker/config.json
Docker Pull User:
I0925 12:04:26.541604    1559 docker.go:503] Using locally available image "cchet/openshift-app-camel:0.1"
I0925 12:04:26.545945    1559 docker.go:503] Using locally available image "cchet/openshift-app-camel:0.1"
I0925 12:04:26.545961    1559 docker.go:734] Image sha256:db059d017558108f65d35825c3cca55694fcbcfecffc6167a332931d6a02d19f contains io.openshift.s2i.scripts-url set to "file:///s2i"
I0925 12:04:26.545971    1559 scm.go:21] DownloadForSource https://github.com/cchet-thesis-msc/app-camel
I0925 12:04:26.545982    1559 scm.go:28] return from ParseFile file exists false proto specified false use copy false
I0925 12:04:26.546007    1559 sti.go:199] Preparing to build
I0925 12:04:26.546215    1559 clone.go:58] Downloading "https://github.com/cchet-thesis-msc/app-camel" ...
I0925 12:04:26.546224    1559 clone.go:62] Cloning sources into "/tmp/s2i247666390/upload/src"
Already on 'master'
Your branch is up-to-date with 'origin/master'.
I0925 12:04:30.072696    1559 clone.go:78] Checked out "master"
I0925 12:04:30.136767    1559 clone.go:84] Updated submodules for "master"
E0925 12:04:30.151454    1559 install.go:243] script "assemble" found by the image URL handler, but failed to install: open /s2i/assemble: no such file or directory
E0925 12:04:30.151483    1559 install.go:243] script "run" found by the image URL handler, but failed to install: open /s2i/run: no such file or directory
I0925 12:04:30.151499    1559 cleanup.go:33] Removing temporary directory /tmp/s2i247666390
I0925 12:04:30.151508    1559 fs.go:236] Removing directory '/tmp/s2i247666390'
I0925 12:04:30.152899    1559 main.go:165] Build failed
E0925 12:04:30.152908    1559 errors.go:266] An error occurred: failed to install [assemble run]
E0925 12:04:30.152912    1559 errors.go:267] Suggested solution: provide URL with Source-To-Image scripts with -s flag or check the image if it contains io.openshift.s2i.scripts-url label set
E0925 12:04:30.152920    1559 errors.go:273] If the problem persists consult the docs at https://github.com/openshift/source-to-image/tree/master/docs. Eventually reach us on freenode #openshift or file an issue at https://github.com/openshift/source-to-image/issues providing us with a log from your build using --loglevel=3
The following is from my docker build command:
Step 12/14 : RUN chmod -R 777 ${S2I_HOME}     && chown -R ${JBOSS_USER}:${JBOSS_USER} ${S2I_HOME}     && echo "Script home: ${S2I_HOME}"     && ls ${S2I_HOME} -lrta
---> Running in dc129334ec44
Script home: /s2i
total 40
-rwxrwxrwx. 1 jboss jboss 2144 Sep 25 09:01 assemble
drwxrwxrwx. 1 jboss jboss 4096 Sep 25 09:46 repository
-rwxrwxrwx. 1 jboss jboss   81 Sep 25 09:56 run
drwxrwxrwx. 1 jboss jboss 4096 Sep 25 09:56 .
drwxr-xr-x. 1 root  root  4096 Sep 25 10:08 ..
---> 6917e82e7ebb
Removing intermediate container dc129334ec44
Step 13/14 : USER ${JBOSS_USER}
---> Running in 9a0b51e08bce
---> b7bf70030ccf
Removing intermediate container 9a0b51e08bce
Step 14/14 : EXPOSE 8080
---> Running in 93026b366c9f
---> 87c10215e5d7
Removing intermediate container 93026b366c9f
Successfully built 87c10215e5d7
Docker labels of builder image
ENV S2I_HOME="/s2i"
S2I_DESTINATION="/s2i/repository"
CAMEL_DATA_DIR="${JBOSS_USER_HOME}/data"
CAMEL_POSTGRES_DRIVER_VERSION="${ARG_VERSION_POSTGRES_DRIVER}"
LABEL io.k8s.description="The wildfly backed camel JEE application"
io.k8s.display-name="App Camel JEE"
io.openshift.expose-services="8080:http"
io.openshift.tags="builder,app-camel"
io.openshift.s2i.destination="${S2I_HOME}/repository"
io.openshift.s2i.scripts-url="file://${S2I_HOME}"

Used wrong URL scheme file:// instead of image://.


Support for andymass/vim-matchup

feat: add support for matchup!
feat: matchup for loops!

This seems to have been done:
https://github.com/andymass/vim-matchup/commit/2550178c43464134ce65328da458905f70dbe041
https://github.com/andymass/vim-matchup/issues/334

Thanks for pointing this out!
Ill have to look, I have a branch here with that supports more features
(like matching arbitrary quote delimiters like qw//), so perhaps I'll send
a PR there
On Tue, Dec 19, 2023, 6:22 PM Seth Daniel @.***> wrote:

This seems to have been done:
@.***
https://github.com/andymass/vim-matchup/commit/2550178c43464134ce65328da458905f70dbe041
andymass/vim-matchup#334
https://github.com/andymass/vim-matchup/issues/334
—
Reply to this email directly, view it on GitHub
https://github.com/tree-sitter-perl/tree-sitter-perl/pull/106#issuecomment-1863080327,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFURPKSECC7KCMNT2GT3UU3YKG5LXAVCNFSM6AAAAAA3KM44G2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRTGA4DAMZSG4
.
You are receiving this because you authored the thread.Message ID:
@.***>

I have a WAY better approach here in #192


Support text on shapes/ROIs and other features

Thanks for your really nice plugging.
I was thinking if would be possible to extend the functionality of this plugin by allowing to add custom text as some people have done here or here. I have been trying to reproduce/adapt those example in our application without success yet :/
Furthermore, would it be possible to export the ROIs with text for later use, that is to read such ROIs coordinates AND the associated text? For instance, imagine the user wanted to create ROIs that want to store and later use with their given text, ID, etc.
I know this may be beyond the scope of the plugin but I think it would enrich a lot the functionality.
A last suggestion would be for instance to add ROIs of fixed shape, for example by a keybind and read the cursor coordinates to position the given ROI with the size value determined by the user.
Sorry if this may looks like a lot. If you could at least guide me how to do couple of this features I can try to also help to implement this features.
Best

Hi @rjlopez2 , thank you for your suggestions!

I was thinking if would be possible to extend the functionality of this plugin by allowing to add custom text

That'll be a nice feature. Yes, I think I can implement this.

would it be possible to export the ROIs with text for later use

Isn't that implemented on the napari side? There's a "save selected layer(s)" command in the file menu (I don't usually save layers so I might have missed something). But eitherway, the napari-style format of shapes layer is not good for saving ROIs. Maybe we need another format (json or parquet?) to do this.

to add ROIs of fixed shape, for example by a keybind and read the cursor coordinates to position the given ROI with the size value determined by the user

Is it like (1) rectangles with fixed width and height, or (2) rectangles with all the coordinates fixed? If (1), I totally agree that we need this feature. In the case of (2), I don't know if it's possible at all,

hi @hanjinliu,
thanks for your reply :)

That'll be a nice feature. Yes, I think I can implement this.

nice 🙌

Isn't that implemented on the napari side? There's a "save selected layer(s)" command in the file menu (I don't usually save layers so I might have missed something). But eitherway, the napari-style format of shapes layer is not good for saving ROIs. Maybe we need another format (json or parquet?) to do this.

yes, that seems to be the case and I agree. Saving ROIs with text within napari is currently not preserving the additional features (color, text, etc). I am not familiar with parquet. I guess a json file would be good to start off.

Is it like (1) rectangles with fixed width and height, or (2) rectangles with all the coordinates fixed? If (1), I totally agree that we need this feature. In the case of (2), I don't know if it's possible at all,

I was thinking at the moment more like 1): rectangles with fixed width and height. I would guess based on this other pluggin  option 2 would be also possible when knowing the coordinate and dimensions of the ROI (Talking about square ROIs only) but for now I don't see an obvious use case except for programmatically creating ROIs at specifics coordinates with specifics sizes.
Thanks a lot and let me know if I can help anyhow.
cheers.

Hey @hanjinliu
just curious if you had some news/updates from this from that i can test.
Cheers!

Hi @rjlopez2 ,
thank you for reminding me of this. I totally forgot this after some busy days... sorry for the inconvenience.
As for the text support and saving/loading the registered ROIs, the current PR will address that (almost done) and I'll finish it this week.
For the user specification of the ROI size, I think it will take longer as it needs another small widget that I've not made yet.

No worries at all.
Awsome!
I haven't tested yet what the current changes are. I will then tested and wait  for the user-defined ROI size feature to arrive.
thanks a lot a gain.

Hi @rjlopez2 ,
Just want to let you know that I've just released v0.0.3, which includes the custom text support (the name of each ROI can be edited, displayed and saved) and importing/exporting json files. Could you try this for now?

Hey @hanjinliu,
Thanks a lot for the effort. I was disconnected for the last few weeks but now I had the chance to quickly have an overview on the changes.
My observations:

the saving and loading options are working fine. It displays the names as they were saved nicely. I see, however, that when you save a ROI set they append an extra semicolon at the end of the json extesion. So this is spotted here here. Removing the semicolon between .json and  *.txt will fix this issue.
I see a new "ROI text" box with the id/name combobox and font size, however I cannot see the text displaying overlayed on the ROI as here for instance. I am not sure how is the way to activate it or if is done by default? may be a checkbox to allow the user to see it would help. Nevertheless, clicking them from the list of ROIs table will activate and highlight the selection, so it helps the user on witch ROI you are working on.
I found that when for instance, you have or load a ROI set and select from the ROI table one you want to remove, the roi is removed  as observed in the viewer but the table does not update. Is that a feature expected or not sure if that's a bug.

I think that all from my side, from now. If you cannot reproduce some of this finding let me know and I'll provide you specifics reprex.
here info about my environment:
>>> utils.sys_info()
'napari: 0.5.4rc4.dev7+ga243d4f6\nPlatform: macOS-14.6.1-x86_64-i386-64bit\nSystem: MacOS 14.6.1\nPython: 3.11.10 | packaged by conda-forge | (main, Sep 30 2024, 17:57:16) [Clang 17.0.6 ]\nQt: 5.15.8\nPyQt5: 5.15.9\nNumPy: 1.26.4\nSciPy: 1.14.1\nDask: 2024.9.1\nVisPy: 0.14.3\nmagicgui: 0.9.1\nsuperqt: 0.6.7\nin-n-out: 0.2.1\napp-model: 0.3.0\npsygnal: 0.11.1\nnpe2: 0.7.7\npydantic: 2.9.2\n\nOpenGL:\n  - GL version:  2.1 ATI-5.5.17\n  - MAX_TEXTURE_SIZE: 16384\n  - GL_MAX_3D_TEXTURE_SIZE: 16384\n\nScreens:\n  - screen 1: resolution 3008x1269, scale 2.0\n\nOptional:\n  - numba not installed\n  - triangle not installed\n  - napari-plugin-manager not installed\n\nSettings path:\n  - /Users/rubencito/Library/Application Support/napari/roi_plugin_test_cde0085a938d7fdcb08e490ff901e8898cd35b67/settings.yaml'

>>> pip list
Package                       Version                 Editable project location
----------------------------- ----------------------- --------------------------------------------------------------------------------------------------------------------------------------------
alabaster                     1.0.0
annotated-types               0.7.0
anyio                         4.6.0
app-model                     0.3.0
appdirs                       1.4.4
appnope                       0.1.4
argon2-cffi                   23.1.0
argon2-cffi-bindings          21.2.0
arrow                         1.3.0
asciitree                     0.3.3
asttokens                     2.4.1
async-lru                     2.0.4
attrs                         24.2.0
Babel                         2.14.0
beautifulsoup4                4.12.3
bleach                        6.1.0
Brotli                        1.1.0
build                         1.2.2.post1
cached-property               1.5.2
cachey                        0.2.1
certifi                       2024.8.30
cffi                          1.17.1
charset-normalizer            3.3.2
click                         8.1.7
cloudpickle                   3.0.0
comm                          0.2.2
contourpy                     1.3.0
cycler                        0.12.1
dask                          2024.9.1
debugpy                       1.8.6
decorator                     5.1.1
defusedxml                    0.7.1
docstring_parser              0.16
docutils                      0.21.2
entrypoints                   0.4
exceptiongroup                1.2.2
executing                     2.1.0
fasteners                     0.19
fastjsonschema                2.20.0
filelock                      3.16.1
flatbuffers                   24.3.25
flexcache                     0.3
flexparser                    0.3.1
fonttools                     4.54.1
fqdn                          1.5.1
freetype-py                   2.5.1
fsspec                        2024.9.0
h11                           0.14.0
h2                            4.1.0
h5py                          3.12.1
HeapDict                      1.0.1
hpack                         4.0.0
hsluv                         5.0.4
httpcore                      1.0.6
httpx                         0.27.2
hyperframe                    6.0.1
idna                          3.10
imagecodecs                   2024.9.22
imageio                       2.35.1
imagesize                     1.4.1
importlib_metadata            8.5.0
importlib_resources           6.4.5
in-n-out                      0.2.1
ipykernel                     6.29.5
ipympl                        0.9.4
ipython                       8.28.0
ipython_genutils              0.2.0
ipywidgets                    8.1.5
isoduration                   20.11.0
jedi                          0.19.1
Jinja2                        3.1.4
json5                         0.9.25
jsonpointer                   3.0.0
jsonschema                    4.23.0
jsonschema-specifications     2024.10.1
jupyter_client                8.6.3
jupyter_core                  5.7.2
jupyter-events                0.10.0
jupyter-lsp                   2.2.5
jupyter_server                2.14.2
jupyter_server_terminals      0.5.3
jupyterlab                    4.2.5
jupyterlab_pygments           0.3.0
jupyterlab_server             2.27.3
jupyterlab_widgets            3.0.13
kiwisolver                    1.4.7
lazy_loader                   0.4
locket                        1.0.0
macro-kit                     0.4.9
magicgui                      0.9.1
markdown-it-py                3.0.0
MarkupSafe                    3.0.1
mat73                         0.65
matplotlib                    3.9.2
matplotlib-inline             0.1.7
mdurl                         0.1.2
mistune                       3.0.2
monochrome-viewer             2024.3.21
mpl-pan-zoom                  1.0.0
munkres                       1.1.4
napari                        0.5.4rc4.dev7+ga243d4f6
napari-console                0.1.0
napari-macrokit               0.0.1
napari-mat-file-reader        0.0.2
napari-matplotlib             3.0.0
napari-omaas                  0.1.3                   /Users/rubencito/Library/Mobile Documents/com~apple~CloudDocs/programming_stuff/p_experimental/image_analysis/napari_play/napari-omaas
napari-plugin-engine          0.2.0
napari-roi-manager            0.0.3                   /Users/rubencito/Library/Mobile Documents/com~apple~CloudDocs/programming_stuff/p_experimental/image_analysis/napari_play/napari-roi-manager
napari-svg                    0.2.0
napari-tiff                   0.1.3
nbclient                      0.10.0
nbconvert                     7.16.4
nbformat                      5.10.4
nest_asyncio                  1.6.0
networkx                      3.3
notebook_shim                 0.2.4
npe2                          0.7.7
numcodecs                     0.13.0
numpy                         1.26.4
numpydoc                      1.8.0
opencv-contrib-python         <IP_ADDRESS>
opencv-python-headless        <IP_ADDRESS>
opticalmapping                0.2.2
overrides                     7.7.0
packaging                     24.1
pandas                        2.2.3
pandocfilters                 1.5.0
parso                         0.8.4
partd                         1.4.2
patsy                         0.5.6
pexpect                       4.9.0
pickleshare                   0.7.5
pillow                        10.4.0
Pint                          0.24.3
pip                           24.2
pkgutil_resolve_name          1.3.10
platformdirs                  4.3.6
ply                           3.11
pooch                         1.8.2
prometheus_client             0.21.0
prompt_toolkit                3.0.48
psutil                        6.0.0
psygnal                       0.11.1
ptyprocess                    0.7.0
pure_eval                     0.2.3
pyconify                      0.1.6
pycparser                     2.22
pydantic                      2.9.2
pydantic-compat               0.1.2
pydantic_core                 2.23.4
Pygments                      2.18.0
pyobjc-core                   10.3.1
pyobjc-framework-Cocoa        10.3.1
PyOpenGL                      3.1.7
pyparsing                     3.1.4
pyproject_hooks               1.2.0
PyQt5                         5.15.9
PyQt5-sip                     12.12.2
PySocks                       1.7.1
python-dateutil               2.9.0
python-json-logger            2.0.7
pytz                          2024.1
PyYAML                        6.0.2
pyzmq                         26.2.0
qtconsole                     5.6.0
QtPy                          2.4.1
referencing                   0.35.1
requests                      2.32.3
rfc3339-validator             0.1.4
rfc3986-validator             0.1.1
rich                          13.9.2
rpds-py                       0.20.0
scikit-image                  0.24.0
scikit-video                  1.1.11
scipy                         1.14.1
seaborn                       0.13.2
seasonal                      0.3.1
Send2Trash                    1.8.3
setuptools                    75.1.0
shellingham                   1.5.4
sif_parser                    0.3.5
sip                           6.7.12
six                           1.16.0
sniffio                       1.3.1
snowballstemmer               2.2.0
soupsieve                     2.5
Sphinx                        8.0.2
sphinxcontrib-applehelp       2.0.0
sphinxcontrib-devhelp         2.0.0
sphinxcontrib-htmlhelp        2.1.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          2.0.0
sphinxcontrib-serializinghtml 2.0.0
stack-data                    0.6.2
static-ffmpeg                 2.7
statsmodels                   0.14.4
superqt                       0.6.7
tabulate                      0.9.0
terminado                     0.18.1
tifffile                      2024.9.20
tinycss2                      1.3.0
toml                          0.10.2
tomli                         2.0.2
tomli_w                       1.1.0
toolz                         1.0.0
tornado                       6.4.1
tqdm                          4.66.5
traitlets                     5.14.3
typer                         0.12.5
types-python-dateutil         <IP_ADDRESS>41003
typing_extensions             4.12.2
typing-utils                  0.1.0
tzdata                        2024.2
uri-template                  1.3.0
urllib3                       2.2.3
vispy                         0.14.3
wcwidth                       0.2.13
webcolors                     24.8.0
webencodings                  0.5.1
websocket-client              1.8.0
wheel                         0.44.0
widgetsnbextension            4.0.13
wrapt                         1.16.0
zarr                          2.18.3
zipp                          3.20.2
zstandard                     0.23.0

I used the following configuration file to test the plugin (I use µmamba as dep/env manager), feel free to ignore many packages here:
name: roi_plugin_test
channels:
  - conda-forge
dependencies:
  - python=3.11
  - pip
  - "numpy<2" # cupy requires numpy<2
  - jupyterlab
  - ipykernel
  - nodejs
  - matplotlib
  - ipympl
  - ipywidgets
  - jupyterlab_widgets
  # - "napari[all]"
  - pyqt
  - seaborn
  # - m2-base # add this for windows compatibility command shell commands
  - pip:
    - opencv-python-headless
    - opencv-contrib-python
    - git+https://github.com/napari/napari.git # this is only to test the head
    - git+https://github.com/rjlopez2/napari-omaas.git
    - git+https://github.com/hanjinliu/napari-roi-manager.git

Again, thanks a lot for the contribution and let me know if you need additional details, test or examples i can provide.
cheers!

Hi @rjlopez2 , thank you for your quick review! Sorry for the late response.

I cannot see the text displaying overlayed on the ROI

There's a "display text" check box in the napari built-in layer control, so I didn't add the same thing to the ROI manager. But I agree that the same check box should be in the ROI manager.

I found that when for instance, you have or load a ROI set and select from the ROI table one you want to remove, the roi is removed as observed in the viewer but the table does not update. Is that a feature expected or not sure if that's a bug.

I think this is a bug. Could you give me a step-by-step example so that I can precisely reproduce on my side?
I'll let you know when things are done 😃


Update precision of values in expressions and restrictions

The precision is currently set to 2, which is not sufficient for some cases, and should therefor be increased.

There are still cases where precision of stored numbers must be increased:

RestrictionDao.minimumNumberValue
RestrictionDao.minimumNumberValue


Fixed setup parameter for windows

Fixed parameter for packaging on windows.

@pearu could you review that?


Add 'Customizing random values with Arbitrary' section for kor doc

Summary
related to #864
translate Customizing random values with Arbitrary section in Customizing Objects part into Korean

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@SooKim1110 추가로 수정할 부분이 있을까요?


Reference system procedures in synapse dedicated pool project

MSSQL Extension Version: 1.16.0
VSCode Version: 1.74.2
OS Version: Windows 10 Enterprise

How to reference system stored procedure like sp_executesql ?
I'm getting this warning for my stored procedure which uses sp_executesql :
Build warning SQL71502: SqlProcedure: [meta].[usp_test] contains an unresolved reference to an object. Either the object does not exist or the reference is ambiguous because it could refer to any of the following objects: [dbo].[sp_executesql] or [meta].[sp_executesql].

I added a reference to the master database but that doesn't solve the warning, the project file looks like:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build">
  <Sdk Name="Microsoft.Build.Sql" Version="0.1.3-preview" />
  <PropertyGroup>
    <Name>synapse</Name>
    <ProjectGuid>{8688EAA4-F8D6-4AB6-9122-F4EC5F406D3B}</ProjectGuid>
    <DSP>Microsoft.Data.Tools.Schema.Sql.SqlDwDatabaseSchemaProvider</DSP>
    <ModelCollation>1033, CS</ModelCollation>
    <DefaultCollation>SQL_Latin1_General_CP1_CS_AS</DefaultCollation>
  </PropertyGroup>
  <Target Name="BeforeBuild">
    <Delete Files="$(BaseIntermediateOutputPath)\project.assets.json" />
  </Target>
  <ItemGroup>
    <ArtifactReference Include="./_build/AzureDw_master.dacpac">
      <SuppressMissingDependenciesErrors>False</SuppressMissingDependenciesErrors>
      <DatabaseVariableLiteralValue>master</DatabaseVariableLiteralValue>
    </ArtifactReference>
  </ItemGroup>
</Project>

I tried to reference sys.sp_executesql (which actually "works" in synapse dedicated pool), but which the compiler still doesn't like:
stdout:  Build warning SQL71502: SqlProcedure: [meta].[usp_test] has an unresolved reference to object [sys].[sp_executesql].

I found this reference from visual studio 2019 where apparently a bug was solved back in 2020, but it doesn't seem to work for me in vscode mssql extension: https://developercommunity.visualstudio.com/t/Warning-in-VS2019-:-object-does-not-exis/824096.
That same page hints that including a "master" dacpac doesn't actually work for synapse all together.

transferred this issue to the DacFx repo because this isn't ADS sql projects specific

Ok got it. Can you try this scenario in SSDT and see if there are any errors there? The sql projects extension in ADS and SSDT should be using the same master dacpacs, but it's possible the one in ADS needs to be updated.

I can help out here: I am getting the same warnings in SSDT for sp_executesql. Database reference for master db is contained in the project. It also doesn't matter if i write dbo.sp_executesql or sys.sp_executesl (that was how i used it in SQL Server usually).
Warning	SQL71502: Procedure: [dbo].[create_table_definition] has an unresolved reference to object [sys].[sp_executesql].	[...] 89 IntelliSense.
Severity	Code	Description	Project	Path	File	Line	Source	Suppression State
Warning	SQL71502: Procedure: [dbo].[prc_sqldw_create_stats] has an unresolved reference to object [dbo].[sp_executesql].	[...] 76 IntelliSense	.

But it's just a warning. Build process works nonetheless.

Any update on this issue, im having the same issue.

@jmetzach Other nuget packages!

Unfortunaly, after reading up and trying it out, it's still not working.
I added the packagereference using the below entry:
dotnet add package Microsoft.SqlServer.Dacpacs.Synapse --version 130.0.0
The project file now looks like:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build">
  <Sdk Name="Microsoft.Build.Sql" Version="0.1.9-preview" />
  <PropertyGroup>
    <Name>synapse</Name>
    <ProjectGuid>{8688EAA4-F8D6-4AB6-9122-F4EC5F406D3B}</ProjectGuid>
    <DSP>Microsoft.Data.Tools.Schema.Sql.SqlDwDatabaseSchemaProvider</DSP>
    <ModelCollation>1033, CS</ModelCollation>
    <DefaultCollation>SQL_Latin1_General_CP1_CS_AS</DefaultCollation>
  </PropertyGroup>
  <Target Name="BeforeBuild">
    <Delete Files="$(BaseIntermediateOutputPath)\project.assets.json" />
  </Target>
  <ItemGroup>
    <PackageReference Include="Microsoft.SqlServer.Dacpacs.Synapse" Version="130.0.0" />
  </ItemGroup>
</Project>

But I'm still getting the same warnings. I know they're only warnings, but I have many such cases so it is a problem for us because it's hard to distinguish from other warnings which I want to know about (and stop the build for)
Example warning when using sys.sp_executesql syntax:
C:\src\synapse\synapsedb\stored_procedures\meta.usp_merge_to_ods.sql(533,10,533,10): Build warning
 SQL71502: SqlProcedure: [meta].[usp_merge_to_ods] has an unresolved reference to object [sys].[sp_executesql]. [C:\src\synapse\synapsedb\synapsedb.sqlproj]

Example warning when omittingthe schema (which is perfectly fine as well):
C:\src\synapse\synapsedb\stored_procedures\meta.usp_delete_landing_data.sql(34,7,34,7): Build warn
ing SQL71562: SqlProcedure: [meta].[usp_delete_landing_data] has an unresolved reference to object [master].[sys].[sp_executesql]. [C:\src\synapse\synapsedb\synapsedb.sqlproj]

@jbtcode I took a look at the master.dacpac for Synapse and noticed it's missing some elements. It's actually missing all of the extended procedures including sp_executesql. We'll work on getting those added correctly.

We have released a new Nuget package containing the latest master.dacpac, give this a try: https://www.nuget.org/packages/Microsoft.SqlServer.Dacpacs.Synapse.Master

Does the docs need an Update?

Yes we'll update the docs shortly.

Hi, this solved part of the problem, the references to sp_executesql and other procedures are now working.
However, the dacpac in this nupkg is still missing quite some objects, which did exist in the previous dacpac (the one I found in https://www.nuget.org/packages/Microsoft.SqlServer.Dacpacs.Master)
For example the systems views (which I need to use to query the synapse request executions information) are missing:

dm_pdw_request_steps and
dm_pdw_request_steps

... but those are far from the only ones, the "old" dacpac is about four times the size of the new one.
I thought, maybe the idea is to use both, but then that could cause conflicts as well I presume?


Documentation:  FMQs (baskets) being released by FDA

Please select a category the issue is focused on?
User Guides
Let us know where something needs a refresh or put your idea here!
Should we add something to admiral to show how to create the FMQs, and add to the ADAE dataset. Please see https://www.regulations.gov/document/FDA-2022-N-1961-0001 and download file to see spreadsheet containing FMQs.
I would suggest creating a Vignette showing how to create an AAG VAD using admiral functions, and using the FMQs supplied by FDA as example custom baskets. We could store just a subset of the FMQs in admiral repo, and make it clear admiral is not storing the FMQs for use, and users should go to the FDA website, to download the latest FMQs if they need them.
Within the Vignette we could also show the calls required in ADAE domain to get the flags from AAG added to ADAE VAD

@bms63 I added some more detail above about what we could do, let me know if this is sufficient.

Do we need to investigate if FMQs are still used/provided by FDA?
The Roche standard group wondered why the MedDRA version is not stated in the document and why there are no frequent updates of the FMQs for each new MedDRA version.
The FMQs are defined in a document labelled as workshop material. I couldn't find an official page regarding FMQs.

@pharmaverse/admiral and @pharmaverse/admiral_comm anyone got any leads on FMQs?

FMQs are still being developed by the FDA- they have currently defined 104 FMQs, and expect at least 30 more in their next release.  There is some information on FMQs from January 2023 posted on the FDA website, at https://www.fda.gov/media/164639/download, though the draft version of the latest FMQs seems to have disappeared.  They did give an update on FMQs at DIA earlier this year, confirming that they are still under active development.
The ADaM team is also working on analysis dataset structures for incorporating FMQs.  The FMQ sub-team presented a poster at the 2023 CDISC Interchange on FMQs, which does not appear to be available online.  Please contact me if you'd like a copy, and I can put you in touch with the author, Guowei Wu.
Thanks,
Nancy

@nbrucken17 thanks for the input - would be good to see the poster to see if there are any possibilities for {admiral} to provide guidance.

I feel like this is more of a discussion rather an issue.  going to move over there.


Added wasabi to S3 cloud storages

Just tried Wasabi cloud storage and it works fine and is easy to setup.

Thank you for the PR 💜.


  This is by analogy with the select statement, in which a case with a nil channel is ignored.  In particular, these four code snippets all block forever:
var c1 chan int
<-c1

var c2 chan int
select {
case <-c2:
}

var c3 chan int
c3 <- 1

var c4 chan int
select {
case c4 <- 1:
}

However, the fact is that send statements and receive expressions are not the same as select cases.  They look different in the program and they are implemented differently.  While it is nice to have a send or receive operation behave identically to a select statement with a single case that corresponds to that operation, it is not necessary, and I claim that changing that correspondence will not cause confusion.
I claim further that the fact that nil channels block forever is a point of confusion for new Go programmers.  They must learn to call make to create a channel.  When learning Go, it is easy to forget that.  This then leads to confusion when programs simply block rather than reporting an error.  Sample comments:
https://groups.google.com/d/msg/golang-nuts/QltQ0nd9HvE/mAb2K4UHWxUJ
https://groups.google.com/d/msg/golang-nuts/JFTymqVDBcs/E8GDJiQFBQAJ
http://devs.cloudimmunity.com/gotchas-and-common-mistakes-in-go-golang/index.html#using_nil_ch
My final claim is that essentially no real programs rely on the current behavior for nil channels.  I have no way to prove this claim, but in general blocking forever is a goroutine leak, and when people do want to block forever anyhow they use select {}.
Give the above, I recommend that for Go 2 we change this corner case and make send or receive operations on nil channels panic.  To be clear I am not proposing that we change the behavior of select statements with nil channels, and I am not proposing that we change any aspect of the behavior for closed channels.  In the above four code snippets, the first and third will panic, while the second and fourth will continue to behave as in Go 1.

To be clear I am not proposing that we change the behavior of select statements with nil channels

I like your proposal and would be happy to see it adopted as-is.
That said, I would go a step further and change the behavior of select statements to match. The use-case for selecting on a nil channel is to toggle whether a particular case can be taken, but it's easy enough to do that with a non-nil channel that has no receivers (to block send cases) or no senders (to block receive cases), and I think the benefit of keeping select and non-select channel ops consistent may outweigh the convenience of nilling out channels to toggle cases.

I think it's good for the statement to be analogous to the select/case except where it needs to be different, and I find blocking on select/case very useful. A lot of times, I will write a loop with a select inside of it, and being able to "turn off" part of the select block with a nil is very useful. See my blog post about this pattern: https://blog.carlmjohnson.net/post/share-memory-by-communicating/

How about panic only when sending to a nil channel? That way we match select and receive behavior and keep the feature of select that it ignores nil channels. I can't give an example right now but I remember it being very useful many times. It's also a common pattern for me to write a loop with select in it that has cases I can toggle on and off.
Also, making select panic would break existing Go 1 code. Not only break but require a complex task of redesigning the application to work around the new behavior. Some cases could be simple but, still, no "go fix" could fix that.

This would effectively remove the possibility of "pausing" a channel as explained in Advanced Concurrency Patterns


WIP: Potential fixes to type hierarchy

@mkborregaard I'm not sure the hierarchy was ideal as it was. Apart from some small errors, and the fact that there is some temporal stuff still in there, there were some unnecessary abstract types - AbstractGrid can be a subtype of AbstractLocations (replacing AbstractPoints), and then AbstractPlaces can have spatial locations or (e.g.) individual names, AbstractLocations can be points, areas or regular points or areas, and AbstractGrid is then specifically regular points or areas. AbstractLocations is better than AbstractPoints because these can be points or areas - we need to setters/getters for that too I guess...

Please feel free to merge this yourself when happy - you've got push rights, no?

Yes, absolutely. I've now fixed those issues, so I'm merging.


Scriggo template engine is not really that fast

I've done some benchmarks, like these:
package hello_test

import (
	"io"
	"testing"

	"github.com/CloudyKit/jet/v6"
	"github.com/open2b/scriggo"
)

func BenchmarkRenderers(b *testing.B) {

    b.Run("jet", func(b *testing.B) {
        l := jet.NewInMemLoader()
        l.Set("lol.html", `
            {{ block Dialog(title) }}

{{ title }}

{{ yield content }}

            {{ end }}
        `)
        l.Set("test.html", `
            {{ import "./lol.html" }}

            {{ yield Dialog(title = "you wrote") content }}
                  woo
            {{ end }}

        `)
        s := jet.NewSet(l)

        t, err := s.GetTemplate("test.html")
        if err != nil { panic(err) }

        for i := 0; i < b.N; i++ {
            err := t.Execute(io.Discard, nil, nil)
            if err != nil { panic(err) }
        }
    })

    b.Run("pure write call", func(b *testing.B) {
        for i := 0; i < b.N; i++ {
            io.Discard.Write([]byte(`

`))
            io.Discard.Write([]byte(`you wrote`))

            io.Discard.Write([]byte(`

`))
            io.Discard.Write([]byte(`
                    woo
            `))

            io.Discard.Write([]byte(`

            `))
        }
    })
    b.Run("scriggo", func(b *testing.B) {
        b.ReportAllocs()

        f := scriggo.Files{
            "test.html": []byte(`
                {% macro Dialog(title string, content html) %}

{{ title }}

{{ content }}

                {% end %}

                {% show Dialog("You wrote", itea); using %}
                  woo
                {% end %}
            `),
        }

        t, err := scriggo.BuildTemplate(f, "test.html", nil)
        if err != nil { panic(err) }

        b.ResetTimer()

        for i := 0; i < b.N; i++ {

            err := t.Run(io.Discard, nil, nil)
            if err != nil { panic(err) }
        }
    })
}

Comparing the same functionality in raw []byte writes, jet template engine and scriggo template.
Now, I don't really know if I'm doing something wrong, as I want to like scriggo, but as a template engine is not really that much faster than text/template. I don't know how it scales for bigger templates.
Is this because it's not yet version 1.0.0?

The benchmarks look correct to me, but we can't expect Scriggo to perform the same as Jet on simple templates because they're designed for different purposes.
Scriggo is a more advanced template system with a scripting language, which means it might be slower due to the extra complexity. On the other hand, Jet is a small template system optimized for quickly generating page layouts without the use of extra scripting.
It's important to consider what you need your template system to do before choosing one, and I'd recommend picking the one that best fits your needs.

Good to know, thanks for the response


dateadd/datediff funcions should be parallel safe ( draft )

task : BABEL-4815
Signed-off-by: Zhibai Song<EMAIL_ADDRESS>Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 72.546%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
41117

Relevant Lines:
56677

💛  - Coveralls


nhanesSearchTableNames

Right now we get:
nhanesSearchTableNames("BMX",details=T)
Questionnaire    TableName     years
1            BMX BodyMeasures 1999-2000
2          BMX_B BodyMeasures 2001-2002
3          BMX_C BodyMeasures 2003-2004
could you add to that the number of rows in the table (nrow in R parlance)
and I don't know if we have this but there is a table name in the HTML
Body Measures (BMX_J)
I think in many cases Nathan has used it to create the TableNames in his DB, but it would be good to have it for every Questionnaire.

Now we do have the number of rows in the table and we will have the details once we have details in the DB.
> nhanesSearchTableNames("BMX",details=T)
   Questionnaire TableName     years
1            BMX       BMX 1999-2000
2          BMX_B     BMX_B 2001-2002
3          BMX_C     BMX_C 2003-2004
4          BMX_D     BMX_D 2005-2006
5          BMX_E     BMX_E 2007-2008
6          BMX_F     BMX_F 2009-2010
7          BMX_G     BMX_G 2011-2012
8          BMX_H     BMX_H 2013-2014
9          BMX_I     BMX_I 2015-2016
10         BMX_J     BMX_J 2017-2018


Update android-studio-canary to <IP_ADDRESS>,171.3829324

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} reports no offenses.
[x] The commit message includes the cask’s name and version.

Thank you for your contribution.


Incorrect return type of roles method in doc block

In HasRoles trait, Doc block of roles method should be \Illuminate\Database\Eloquent\Relations\BelongsToMany.

Thanks very much for your PR!


Research existing data visualization solutions

Data visualization is a possible route we can follow to make our emissions data more accessible. But - why start from scratch? Let's see what existing solutions are out there (and maybe even learn from them!)
The scope of this ticket will be to find examples of existing emissions data visualizations. Though we can no doubt learn from tools that display complex data of all kind, we'll keep it to this limited scope as a result of team capacity.
Acceptance Criteria:

[ ] Research existing digital/interactive emissions data visualizations in Canada
[ ] Research existing digital/interactive emissions data visualizations worldwide
[ ] Compile findings in Miro
[ ] Share findings with project team

Looks good to me @jaimiebutton. To clarify, you want to look at existing tools/systems that display emissions data. Not specific platforms that can be used (e.g, Mapbox, ArcGIS, etc). As part of this and in the miro documentation, it would be good to indicate that you aren't comparing datasets, so our solution would probably differ.

Thanks for that clarification @LindsayMacfarlane - I've modified above. Good call about comparing datasets, though it could still be a recommendation of the project to compare/combine our data with another dataset to be able to tell a wider story. (So perhaps then best to keep it open until we narrow it down)
What are your thoughts?


Moving feature specific dependencies to optional installs in 3.0?

Is your idea related to a problem? Please describe.
There are quite a few issues related to dependencies with the current installation setup. In our organization, the dependency conflicts/issues are getting to the point of considering removing aws-sdk-pandas as a dependency and re-implementing some of the functionality.
I (and likely others) am hoping this situation can be improved upon, because aws-sdk-pandas IS an awesome library! Happy to help make this happen.
Main issues

Heavy load of transitive dependencies

It seems many users might be using one or two connectors/services, but with the current setup, they are getting dependencies for a lot of unused functionality.
Longer download times, even if one is only using one of the connectors/services like s3
Larger attack surface to manage (see below)

Quite restrictive on the dependency versions

Prevents users from updating
Users who follow the best practice of pinning or restricting dependencies at the application level are unable to get bug fixes to critical packages like numpy
I understand there are reasons for restricting to a degree, but the current setup is overly restrictive, mixed with the current release cadence it creates a major roadblock
Maybe consider releasing daily or weekly if it's imperative the dependency restrictions not be relaxed

Describe the solution you'd like
With an upcoming 3.0 release, I wonder if it would be possible to comb though the dependencies and move the specific connector/service related deps to a service-related extra. i.e everything for redshift goes in a "redshift" extra, everything for lakeformation goes in a "lakeformation" extra etc. A major release would be the perfect timing for this IMO.
Dask has a pretty good setup/example of how to handle this and educate users about extras. Maybe consider doing something similar for aws-sdk-pandas?
https://docs.dask.org/en/stable/install.html#pip
Additionally, asking for the dependency restrictions to be relaxed so users can incorporate bug fixes. Mostly around pandas, pyarrow, boto and numpy. It should be up to the user to pin and or upgrade. If downstream isn't following a best practice, educate them on how to.
Transitives
On a fresh venv I end up with the following 52 packages from installing awswrangler with no extras. For comparison, a venv with the "core" requirements (pandas, numpy pyarrow, boto3) installed there are 16 packages. All of these are not needed for interacting with an individual service.
pip list
aenum              3.1.11
aiohttp            3.8.1
aiosignal          1.3.1
asn1crypto         1.5.1
async-timeout      4.0.2
attrs              22.2.0
awswrangler        2.19.0
backoff            2.2.1
beautifulsoup4     4.11.2
boto3              1.26.62
botocore           1.29.62
certifi            2022.12.7
charset-normalizer 2.1.1
decorator          5.1.1
et-xmlfile         1.1.0
frozenlist         1.3.3
gremlinpython      3.6.2
idna               3.4
isodate            0.6.1
jmespath           1.0.1
jsonpath-ng        1.5.3
lxml               4.9.2
multidict          6.0.4
nest-asyncio       1.5.6
numpy              1.23.4
openpyxl           3.0.10
opensearch-py      2.1.1
packaging          23.0
pandas             1.5.1
pg8000             1.29.4
pip                22.3.1
pipdeptree         2.3.3
ply                3.11
progressbar2       4.2.0
pyarrow            10.0.1
PyMySQL            1.0.2
python-dateutil    2.8.2
python-utils       3.4.5
pytz               2022.7.1
redshift-connector 2.0.910
requests           2.28.2
requests-aws4auth  1.2.1
s3transfer         0.6.0
scramp             1.4.4
setuptools         65.6.3
six                1.16.0
soupsieve          2.3.2.post1
urllib3            1.26.14
wheel              0.38.4
yarl               1.8.2

Output from pipdeptree.
awswrangler==2.19.0
  - backoff [required: >=1.11.1,<3.0.0, installed: 2.2.1]
  - boto3 [required: >=1.24.11,<2.0.0, installed: 1.26.62]
    - botocore [required: >=1.29.62,<1.30.0, installed: 1.29.62]
      - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
      - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.2]
        - six [required: >=1.5, installed: 1.16.0]
      - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.14]
    - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
    - s3transfer [required: >=0.6.0,<0.7.0, installed: 0.6.0]
      - botocore [required: >=1.12.36,<2.0a.0, installed: 1.29.62]
        - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
        - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.2]
          - six [required: >=1.5, installed: 1.16.0]
        - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.14]
  - botocore [required: >=1.27.11,<2.0.0, installed: 1.29.62]
    - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
    - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.2]
      - six [required: >=1.5, installed: 1.16.0]
    - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.14]
  - gremlinpython [required: >=3.5.2,<4.0.0, installed: 3.6.2]
    - aenum [required: >=1.4.5,<4.0.0, installed: 3.1.11]
    - aiohttp [required: >=3.8.0,<=3.8.1, installed: 3.8.1]
      - aiosignal [required: >=1.1.2, installed: 1.3.1]
        - frozenlist [required: >=1.1.0, installed: 1.3.3]
      - async-timeout [required: >=4.0.0a3,<5.0, installed: 4.0.2]
      - attrs [required: >=17.3.0, installed: 22.2.0]
      - charset-normalizer [required: >=2.0,<3.0, installed: 2.1.1]
      - frozenlist [required: >=1.1.1, installed: 1.3.3]
      - multidict [required: >=4.5,<7.0, installed: 6.0.4]
      - yarl [required: >=1.0,<2.0, installed: 1.8.2]
        - idna [required: >=2.0, installed: 3.4]
        - multidict [required: >=4.0, installed: 6.0.4]
    - isodate [required: >=0.6.0,<1.0.0, installed: 0.6.1]
      - six [required: Any, installed: 1.16.0]
    - nest-asyncio [required: Any, installed: 1.5.6]
  - jsonpath-ng [required: >=1.5.3,<2.0.0, installed: 1.5.3]
    - decorator [required: Any, installed: 5.1.1]
    - ply [required: Any, installed: 3.11]
    - six [required: Any, installed: 1.16.0]
  - numpy [required: >=1.21.0,<=1.23.4, installed: 1.23.4]
  - openpyxl [required: >=3.0.0,<3.1.0, installed: 3.0.10]
    - et-xmlfile [required: Any, installed: 1.1.0]
  - opensearch-py [required: >=1,<3, installed: 2.1.1]
    - certifi [required: Any, installed: 2022.12.7]
    - requests [required: >=2.4.0,<3.0.0, installed: 2.28.2]
      - certifi [required: >=2017.4.17, installed: 2022.12.7]
      - charset-normalizer [required: >=2,<4, installed: 2.1.1]
      - idna [required: >=2.5,<4, installed: 3.4]
      - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.14]
    - urllib3 [required: >=1.21.1,<2, installed: 1.26.14]
  - pandas [required: >=1.2.0,<=1.5.1,<2.0.0,!=1.5.0, installed: 1.5.1]
    - numpy [required: >=1.21.0, installed: 1.23.4]
    - python-dateutil [required: >=2.8.1, installed: 2.8.2]
      - six [required: >=1.5, installed: 1.16.0]
    - pytz [required: >=2020.1, installed: 2022.7.1]
  - pg8000 [required: >=1.20.0,<2.0.0, installed: 1.29.4]
    - python-dateutil [required: >=2.8.2, installed: 2.8.2]
      - six [required: >=1.5, installed: 1.16.0]
    - scramp [required: >=1.4.3, installed: 1.4.4]
      - asn1crypto [required: >=1.5.1, installed: 1.5.1]
  - progressbar2 [required: >=4.0.0,<5.0.0, installed: 4.2.0]
    - python-utils [required: >=3.0.0, installed: 3.4.5]
  - pyarrow [required: >=2.0.0,<10.1.0, installed: 10.0.1]
    - numpy [required: >=1.16.6, installed: 1.23.4]
  - pymysql [required: >=1.0.0,<2.0.0, installed: 1.0.2]
  - redshift-connector [required: >=2.0.889,<2.1.0, installed: 2.0.910]
    - beautifulsoup4 [required: >=4.7.0,<5.0.0, installed: 4.11.2]
      - soupsieve [required: >1.2, installed: 2.3.2.post1]
    - boto3 [required: >=1.9.201,<2.0.0, installed: 1.26.62]
      - botocore [required: >=1.29.62,<1.30.0, installed: 1.29.62]
        - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
        - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.2]
          - six [required: >=1.5, installed: 1.16.0]
        - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.14]
      - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
      - s3transfer [required: >=0.6.0,<0.7.0, installed: 0.6.0]
        - botocore [required: >=1.12.36,<2.0a.0, installed: 1.29.62]
          - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
          - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.2]
            - six [required: >=1.5, installed: 1.16.0]
          - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.14]
    - botocore [required: >=1.12.201,<2.0.0, installed: 1.29.62]
      - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
      - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.2]
        - six [required: >=1.5, installed: 1.16.0]
      - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.14]
    - lxml [required: >=4.6.5, installed: 4.9.2]
    - packaging [required: Any, installed: 23.0]
    - pytz [required: >=2020.1, installed: 2022.7.1]
    - requests [required: >=2.23.0,<3.0.0, installed: 2.28.2]
      - certifi [required: >=2017.4.17, installed: 2022.12.7]
      - charset-normalizer [required: >=2,<4, installed: 2.1.1]
      - idna [required: >=2.5,<4, installed: 3.4]
      - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.14]
    - scramp [required: >=1.2.0,<1.5.0, installed: 1.4.4]
      - asn1crypto [required: >=1.5.1, installed: 1.5.1]
    - setuptools [required: Any, installed: 65.6.3]
  - requests-aws4auth [required: >=1.1.1,<2.0.0, installed: 1.2.1]
    - requests [required: Any, installed: 2.28.2]
      - certifi [required: >=2017.4.17, installed: 2022.12.7]
      - charset-normalizer [required: >=2,<4, installed: 2.1.1]
      - idna [required: >=2.5,<4, installed: 3.4]
      - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.14]
    - six [required: Any, installed: 1.16.0]

Thank you @jmahlik, I agree with your points. PR #1992 should address this as part of release 3.0

Thank you @jmahlik, I agree with your points. PR #1992 should address this as part of release 3.0

Awesome, just ran our internal test suites off the release-3.0.0 branch and this completely resolves all the issues and passes without breakage :). Even on python 3.11.

Hi @jaidisido, this optional dependency is the last blocker for Python 3.11 support. When will this be available on PyPI as a 3.0.0rc release?

@menaitm I did the workaround from https://github.com/aws/aws-sdk-pandas/issues/1714#issuecomment-1435156127 on 2.x and it worked wonderfully. Pin gremlinpython==3.6.3rc1, assuming you aren't actually using it.


firebase-admin version

Hello,
Would you please upgrade your package.json to use newer version of firebase-admin, now npm shouts:
Step #1: npm WARN deprecated<EMAIL_ADDRESS>This version is no longer maintained. Please upgrade to the latest version.
Step #1: npm WARN deprecated<EMAIL_ADDRESS>This version is no longer maintained. Please upgrade to the latest version.
Step #1: npm WARN deprecated<EMAIL_ADDRESS>This version is no longer maintained. Please upgrade to the latest version.

thanks for the report, fixed on version 1.4.0


Errors after connecting

Hi there.
I'm using the example script, but once I get connected, I start getting spammed this error message.
Error: getEvents(): Error: getaddrinfo ENOTFOUND front6 front6:80
Error: getEvents(): Error: getaddrinfo ENOTFOUND front6 front6:80
Error: getEvents(): Error: getaddrinfo ENOTFOUND front6 front6:80

Am I doing something wrong?

Node v6.10.3

I think it's because Omegle changed the subdomain.

Hi, thanks for creating the issue. I've pushed a fix for that and published the latest version on npm, let me know if that fixes it for you. :)


Improved handling of content strings / errors in the Recipient Tool

Is your feature request related to a problem? Please describe.
Recipient tool is of course critical to the utility of Langroid. While ChatGPT produces picture perfect handling of the recipient tool, other LLMs such as Llama 3.1 are just a bit off. In particular, they can be inclined to put the content in as JSON rather than as a string.  Langroid handles this reasonably gracefully, reminding the LLM to please reformat the request.  But this process is time consuming and doesn't always work as desired.
Describe the solution you'd like
A first pass look at the LLM result which determines that the recipient tool is being called could also modify the content before passing on to the Recipient Tool class itself. Specifically, can you check for non-quoted content and automatically add quotes, change double quotes to single quotes inside or do the necessary escaping inside so that it can be handled by the Recipient Tool without error.
Describe alternatives you've considered
I have tried to change my prompts in many ways as well as the prompt message inside the Recipient Tool to no success.
Additional context
This small change would be immensely helpful in enabling production use where we cannot use commercial LLMs and are limited to slightly weaker models.

@jduke99 thanks for this suggestion. Please if this FAQ (and related example) I just added helps --
https://langroid.github.io/langroid/FAQ/#how-can-i-deal-with-llms-especially-weak-ones-generating-bad-json-in-tools
TLDR - weak LLMs often return bad/hard-to-fix JSON in tools. XML-based tools can be much more reliable. Also SendTool is simpler to use than RecipientTool.

closing/stale


[DO_NOT_MERGE][raz_s3] Give user option to create own S3 user dir

The PR is ready for review. It will be squashed and merged once approved.


How to make the recyclerview swipelayout unswipeable on some event

How to make the recyclerview swipelayout unswipeable on some event

see the com.daimajia.swipe.SwipeLayout.SwipeDenier on how to prevent swipes on specific rows.


[Feature]: Overlapping Widgets

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
Currently, it is not possible to have overlapping widgets in Appsmith. Several users have requested this feature as they would like to use a widget on top of another one and show/hide them conditionally, but today it is only possible to place the widgets next to each other.
Use cases:

Overlap an image widget and a video widget so that if the URL of the asset is an image, the image widget will be displayed and the video widget will be hidden, and vice-versa.
Overlap 2 buttons and based on a condition, show only one of them and hide the other. For this scenario, the workaround is to use a single button and set the label/color/onChange event conditionally using JS.

Why should this be worked on?
This feature will improve the user experience by allowing users to overlap 2 or more widgets and show or hide them conditionally.

Rather than overlap, we'll be enabling this via #11038 where the widget that is hidden will take up no space and thus the widget below it will show there


__docker_daemon_is_experimental: command not found

Description
Bash completion for docker service shows __docker_daemon_is_experimental: command not found
Ubuntu 16.10, Docker Engine 1.13.1
Steps to reproduce the issue:

in terminal type: docker service
press tab

Result:
__docker_daemon_is_experimental: command not found
Expected:
List of available commands should be printed.
Output of docker version:
Client:
 Version:      1.13.1
 API version:  1.26
 Go version:   go1.7.5
 Git commit:   092cba3
 Built:        Wed Feb  8 06:57:21 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.1
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   092cba3
 Built:        Wed Feb  8 06:57:21 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:
Containers: 8
 Running: 2
 Paused: 0
 Stopped: 6
Images: 73
Server Version: 1.13.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 153
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: active
 NodeID: dxtd2cwx56h2rnt3m0l26fzqm
 Is Manager: true
 ClusterID: 3tk9zsy96qd1cbnscb94cpp84
 Managers: 1
 Nodes: 1
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
 Node Address: <IP_ADDRESS>
 Manager Addresses:
  <IP_ADDRESS>:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.8.0-37-generic
Operating System: Ubuntu 16.10
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.54 GiB
Name: ********************
ID: G244:BV5P:3LAX:PHPJ:DVPH:GHYC:RC2X:AZLG:JSTN:EQEI:HLVP:W7LL
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

@albers PTAL?

@pszczekutowicz Please post the output of ls -l /etc/bash_completion.d/docker and md5sum /etc/bash_completion.d/docker.

@pszczekutowicz Did you do a clean install or an upgrade of Docker?
If you upgraded, please tell me your upgrade history, beginning from the last 12.x release.

I can reproduce, will investigate further.

Sorry, I was unable to respond earlier. Thanks for the fix.

@pszczekutowicz The issue will be fixed in a potential 1.13.2 release.
See https://github.com/docker/docker/pull/30875#issuecomment-279323995 for how to get a fixed bash completion script.


Strategy for rendering bloq_counts from a CirqGateAsBloq object?

Currently rendering something like QROM as a nice bloq_counts picture requires a bit of hackery (see below).
What I have done below also seems wrong, our bloqs are frozen but I'm overwriting their bloq counts on the fly (which is somewhat related to the question of how one would modify the decomposition of an existing bloq but with one primitive substituted out.) I could have defined a bloq_counts on QROM alternatively but this seems to defeat the purpose of automatically determining the counts from the decomposition.
import numpy as np
from cirq_ft.algos import QROM
from qualtran.cirq_interop import CirqGateAsBloq

from qualtran.resource_counting import get_bloq_counts_graph, GraphvizCounts
from qualtran.bloqs.util_bloqs import Split, Join, Allocate, Free
from cirq_ft.algos.and_gate import And
import sympy

from typing import Any, Dict, Optional, Set, Tuple

from qualtran import Bloq, Register, Side, Signature, Soquet, SoquetT
from qualtran.bloqs.basic_gates import TGate
from qualtran.bloqs.util_bloqs import ArbitraryClifford
from qualtran.resource_counting import big_O, SympySymbolAllocator

def bloq_counts(self, ssa: Optional['SympySymbolAllocator'] = None) -> Set[Tuple[int, Bloq]]:
    cv1, cv2 = self.gate.cv
    if isinstance(cv1, sympy.Expr) or isinstance(cv2, sympy.Expr):
        pre_post_cliffords = big_O(1)
    else:
        pre_post_cliffords = 2 - cv1 - cv2
    if self.gate.adjoint:
        return {(4 + 2 * pre_post_cliffords, ArbitraryClifford(n=2))}

    return {(9 + 2 * pre_post_cliffords, ArbitraryClifford(n=2)), (4, TGate())}
#
ignore = ['H, XPow', 'Y', 'X', 'Free', 'CNOT', 'T', 'H', 'S', 'Measurement', 'reset']
cirq_ignore = [f'cirq.{g}' for g in ignore]

def generalize(bloq):
    if 'cirq.And' in bloq.pretty_name():
        bloq.__class__.bloq_counts = bloq_counts
    if any([g in bloq.pretty_name() for g in cirq_ignore]):
        return None
    if isinstance(bloq, Allocate):
        return None
    if isinstance(bloq, Free):
        return None
    return bloq

cirq_qrom = QROM.build(*[np.random.randint(0, 10, 100)])
def custom_repr(self):
    selection_repr = repr(self.selection_bitsizes)
    target_repr = repr(self.target_bitsizes)
    return (f"cirq_ft.QROM(selection_bitsizes={selection_repr}, "
        f"target_bitsizes={target_repr}, num_controls={self.num_controls})"
    )
cirq_qrom.__class__.__repr__ = custom_repr
qrom = CirqGateAsBloq(cirq_qrom)
graph, sigma = get_bloq_counts_graph(qrom, generalizer=generalize)
GraphvizCounts(graph).get_svg()

Handling the bloq_counts during decomposition is certainly not the way to go.

A better way maybe better way to do it is
and_cv0 = ssa.new_symbol('cv0')
and_cv1 = ssa.new_symbol('cv1')
def generalize(bloq):
    if ('cirq.And' in bloq.pretty_name()) and (len(bloq.gate.cv) == 2):
        return And(cv1=and_cv0, cv2=and_cv1, adjoint=bloq.gate.adjoint)

@mpharrigan is this last way something sensible, or is there a better way?

yes
strictly speaking you may want a more defensible condition than if "cirq.And" in bloq.pretty_name() but that's details

there's almost no CirqGateAsBloq in the call graphs in the standard library. Otherwise, CirqGateAsBloq should use its __str__ method to control how it is displayed


How to use a VIndex type "hash" in a vschema, need a example of vschema/schema.

As I have my own unique key, and want to use it as "keyspace_id" for sharding.
In example/demo, I see how to create an auto_increase hash key of keyspace. Now I'm seeking the way to use a "hash" key:

https://github.com/youtube/vitess/blob/master/doc/V3VindexDesign.md
numeric: binpack a uint64 into a keyspace_id
hash: hashes a uint64 into a keyspace_id
hash_autoinc: Same as hash, but can generate autoincrement values
lookup_hash: Uses a lookup table, and hashes the result to generate a keyspace_id. It’s non-unique.
lookup_hash_unique: lookup_hash, but unique
lookup_hash_autoinc
lookup_hash_unique_autoinc

For example, use my own user_id(uint64) which is hashed from a unique key.
My testing output:

WARNING:root:vtclient_exception for None.None.master: ('VitessError', u'execInsertSharded: hash.Create: cannot route query: insert into () values(:): syntax error at position 14', 'Execute', 2, "insert into tbl_user(keyspace_id, app_id, account, password, nickname, avatar, gender, signature, extern, device, version) values(1, 'appid1', 'user_1', 'pass_1', 'name_1', '', 0, '', '', '', '') /* run this a few times */", 'tablet_type=master')

My config files:
vschema.zip
Very appreciated to any help~

Sorry about responding late on this. Looks like you closed this issue. Let us know if you still need this question (or others) answered.

Not at all, I have fixed my problems.
I found I should use "numeric" vindex type instead of "hash", and use these vschema, can successfully setup my own keyspace id.
"user_idx": {
"Type": "numeric"
}
this is very simple, but I have a problem before because my user_id column(which used as keyspace_id) is SQL type of BIGINT,  this cause I cannot shard to the second shard.
That should be BIGINT UNSIGNED, then everything runs well.
My user_id which is bigger then or equal to 0x8000 0000 0000 0000 will be sharded to the 80- shard.

This is because v3 is not yet feature-complete on the resharding side.
The way to make this work (for now) is to create both columns: user_id and keyspace_id, and populate them both. The v3 API can actually auto-fill the keyspace_id column based on user_id.
Basically, the non-v3 resharding still requires the keyspace_id column.
Once v3 is feature-complete, resharding will work with just the user_id column.

In my test, I create only a user_id column and no keyspace_id, no problem got by now.
Seems current V3 version will create a internal keyspace_id and use it (I guess).
Thanks for your help~

Another question ~
In a real deployment,
conn, err := vtgateconn.DialProtocol(ctx, "grpc", *server, timeout)
Should I create a connection pool to manage my conns(to vtgate)? Will that benefit something or vitess has done these things for me?

The vtgate connection is multiplexed, and you can send multiple simultaneous requests through it.
These requests eventually reach vttablet, which will use a connection from a pool of mysql connections to run your query.

Thanks~, so I just need keep only one vtgate connection in my client, and it is thread-safe to multiple request Excute/Transactions. For example, I may call vtgate.conn.Execute(use same conn) from concurrency goroutines, is it OK too?
I have another problem, I found I cannot change a vindex column values, that will got:

error: cannot change vindex columns

I think some situations need to change those columns, if a have a user keyspace, and make "name" column a vindex to lookup "user_id", I can use vindex type "lookup_hash". However, user may change their "name" in future, is there some ways to do this change?

Yes. You can execute multiple concurrent requests to VTGate.
The reason why you cannot change a vindex column is because it may result in the row migrating from one shard to another. However, you can achieve the same thing by deleting the row and re-creating it with the new values. This will ensure that every row is always consistent with respect to the keyspace_id.

I can understand if I change the keyspace_id value, may result in the row migrating from one shard to another. But if I only want to change a lookup column, may it result in migrating too?
It seems that "user" keyspace is sharded by "user_id", and "lookup" keyspace is not sharded.
So I think change "name" column should not make effect to "user_id" itself. Only do these changes:

change the row in lookup table(lookup keyspace):  name_user_idx, change "name" column
change the row in user table(user keyspace): change "name" column

I think what you say is possible. I'll need to dig into the code to make sure that allowing this doesn't accidentally allow other things that could potentially be invalid.
I'll let you know what I find.

Thanks a lot, I'll read more code too, and keep tracking this.

I looked at the code. We currently allow lookup tables to be used as a shared foreign key index. This means that multiple tables can reuse an existing name->keyspace_id relationship to co-locate (and find) data that belongs to the same name.
In such situations, changing a name could cause rows from other tables to become unreachable.
We're currently debating whether shared foreign keys are worth it. If we decide to not support this, we can reconsider allowing name changes.
Note that there is a similar issue with deletes. However, it's easier to expect the app to delete all detail table rows before deleting the main table row, which will also remove the lookup entry.

Really appreciate your great help and job!
It is true what you said, and I think using shared foreign key index is for performance considering.
Really useful for me of the details you supply.
During my use of Vitess, I found I often need to restart vtgate progress, that's not good for me.
I often meet this situation:

create a new table of a existing sharded keyspace
I do "ApplySchema" to the keyspace, to create table.
I do "ReloadSchema" to every tablet in that keyspace.
because that table has a new vindex(often it is a auto-incr id and lookup the keyspace_id, If I don't create a vindex for table, I will get gRPCServerError: error: DML too complex Error when insert), I also need to change my vschema.json. So i do "ApplyVSchema" to vtgate.

After all above done, I cannot do a insert to my new table, an error: "table tbl_test not found" faised, although I can found the schema by "GetSchema" command, checked it is correct. At last, I need restart vtgate, then things will go well.
It seems that, if i change vschema, I need to restart vtgate?

It's a known issue. We intend to fix it.


Fixes #216, fixes #170

Still some //todo comments to followup with
Changed ugly LocalDate references to pretty DateTime

looks good. couple of minor comments.
oh and the build... :)


add config toggle to disable any package updates on cloud-init

This adds a config toggle to disable any package install or updates during boot / cloud-init.
This PR belongs to https://github.com/cloudbase/garm/pull/84
Remarks:
I'm no really happy to duplicate this entire function just to add this if.
Michael Kuhnt<EMAIL_ADDRESS>Mercedes-Benz Tech Innovation GmbH (ProviderInformation)

Yeah, you are right. In the future, I think we should make this part a bit more granular to allow easy customization. For now, I think the easiest and quickest thing to do is add an extra param to GetCloudConfig() function in garm itself that disables updates. This way we don't have to copy it just for this.
Feel free to create a PR in garm that adds the flag. We can vendor it here and make use of it.

For now, I think the easiest and quickest thing to do is add an extra param to GetCloudConfig() function
.. or maybe we could add a new field UserDataOptions to the BootstrapInstance struct? This could be the first step towards more customization..

If you remove the replace and update to the latest commit of garm, we can merge this one too

If you remove the replace and update to the latest commit of garm, we can merge this one too

done

Thanks!


adding new aws link under volume sourcing

This is for J hence no Jira ticket. Needed to add AWS link 'https://www.dta.gov.au/help-and-advice/ict-procurement/tools-sourcing-digital-products-and-services/ict-panels-and-arrangements/buying-aws-products' under volume sourcing on the homepage. Its called AWS Products

can you remove the following files from this PR:

package-lock.json
package.json
pancake/css/pancake.min.css

can you not include the following files from this PR:

package-lock.json
package.json
pancake/css/pancake.min.css

Done


stateless: add a test for toDot and fix a commitment copy issue

Codecov Report

Merging #154 (84d4941) into ipa-stateless (137c833) will increase coverage by 2.59%.
The diff coverage is 100.00%.

@@                Coverage Diff                @@
##           ipa-stateless     #154      +/-   ##
=================================================
+ Coverage          82.48%   85.07%   +2.59%
=================================================
  Files                  9        9
  Lines                782      784       +2
=================================================
+ Hits                 645      667      +22
+ Misses               114       96      -18
+ Partials              23       21       -2

Impacted Files
Coverage Δ

stateless.go
71.63% <100.00%> (+9.01%)
:arrow_up:

tree.go
90.23% <0.00%> (+0.46%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 137c833...84d4941. Read the comment docs.


Optional SSL also for ES startup check script

Hey,
I didn't have much time to test it, but it looks like the HTTP_SSL is not covered at this line:
https://github.com/Khezen/docker-elasticsearch/blob/master/src/miscellaneous/wait_until_started.sh#L5
If I turn off the SSL on port 9200 that script will try to use SSL anyway which will fail.

true I'll fix it quickly. Thanks for reporting.


refactor(ast): add walk_mut functions

In this PR I'm going to address the issue of our visitor abstraction using the walk functions similar to how it is implemented in the rustc visitors.
It is the continuation of #2764, and should get merged after it since I've continued on those commits.

If you guys really want it, We can squash this PR instead of having 2 but I think since they are addressing different problems we should keep them seperated.

Since this PR has already exceeded 5k lines of code moving around, I'm going to break it into 2 parts to make merging it less of a chore.
In this PR I've tried to implement walk functions for the VisitMut trait, these functions are contained in their own module called walk_mut. I also moved both visit and visit_mut modules to a super module called visit which shouldn't be really necessary but I find it more pleasant and have no strong preference if you want to revert it.

Thank you for your efforts in accomplishing this amazing work. We are thinking about how VisitMut can be implemented better as it is crucial for us to implement the transformer. btw, @Boshen is on vacation, so he may not respond in time.

Thank you for your efforts in accomplishing this amazing work. We are thinking about how VisitMut can be implemented better as it is crucial for us to implement the transformer. btw, @Boshen is on vacation, so he may not respond in time.

Thanks for letting me know mate, I would also love to see your feedback if you have any on this and/or my other PR.

Love this change.
I do have a suggestion though, maybe in a follow-up PR, but can we rename AstKind or AstType? They basically mean the same thing, and I'm constantly getting confused which is which.

@milesj Hi, Thanks for the feedback.
I have to agree on the fact that having similar names may cause some confusion for users especially if they are new to this codebase. Do you have any suggestions for it? As it is right now they are both needed, AstKind is used for making the semantics in the Visit trait and AstType is a non-referencing version of it for VisitMut which has more limitations on passing references around. If @Boshen green lights it we may be able to do the same thing here for Visit but we are still going to need AstKind for our AstNode type.
I based this naming on something an old collage of mine taught me before, Basically, the conversion here is that AstType is an enum of all types of AST nodes, while AstKind is an enum that contains anything of the AST kind, so one is there to hold the Ast and other one is there to inform us only about the type.

It is ready to merge again!

@Boshen Sorry to ping you here, This PR moves visit traits to their own module. If it is something you are not found of just let me know to revert it before resolving the conflicts here.

I like how things are structured in syn https://docs.rs/syn/latest/syn/index.html#modules
It's all green light from me if the direction we are heading is "be like syn" 😅

I like how things are structured in syn https://docs.rs/syn/latest/syn/index.html#modules
It's all green light from me if the direction we are heading is "be like syn" 😅 mod things so we can feature gate them to reduce a bit of compile time.

Well, I'm not familiar with how syn has structured its code but I gave it a cursory look and I get what you mean, We can definitely add features to selectively choose from toolchain modules.
I mostly moved visitors to their own module to have them in a separate directory in case we want to add things like ScopeVisit and other visitor traits. It shouldn't make any difference in how we want to feature gate them.

Our largest consumer right now is Rolldown, I want to make it compile as fast as possible.

I'll take a look into how Rolldown is using this project to see what features can be feature-gated and which set of them can be grouped together.

I'll give you write permission if you like this project, and intend to push things forward (especially regarding the AST and the transformer) 😁

Well, I'm grateful for the offer, I actually am interested in making the transformer feature complete(at least the preset-env part of it), and having the write permission for stuff that has tests; that won't need that much supervision can indeed smooth things out.
As for the AST and feature gating, it won't make that big of a difference since we almost always need to have you and/or other contributors review the changes.

Let me know what's your decision about moving the visit traits.

Let me know what's your decision about moving the visit traits.

👍

@rzvxa https://github.com/oxc-project/oxc/invitations
I gave you write permission so you get access to graphite. https://oxc-project.github.io/docs/contribute/rules.html

@rzvxa https://github.com/oxc-project/oxc/invitations
I gave you write permission so you get access to graphite. https://oxc-project.github.io/docs/contribute/rules.html

Well thanks, I haven't used graphite before so bear with me in my first few PRs till I get the hang of it.


Multi orientation videos

Hi, I really like your project, thanks for sharing. Im working on a private project where I would like to make the videocomposition landscape for all videos if just one of the videos selected for merge is landscape. and then rotate the rest of the portrait oriented videos accordingly while having black spaces on each side. Will you please point me in the right direction in regards of makeing an addition to you great project. I think this particually part is missing in most video merge projects I've seen...

I'm sorry, I don't really have the time right now to research this and I haven't done it, so I can't really tell you what to do off the top of my head.
But I've seen some Stack Overflow answers that explain how to. You can create a method that rotates all portrait videos to landscape before you use the generator to merge them instead of trying to rewrite the existing code since that would be more difficult. That's the first thing that comes to mind right now.


 IF YOU DO NOT FOLLOW THE FORMAT OR MAKE A GENERIC TOKEN REQUEST YOUR ISSUE WILL BE DELETED WITHOUT COMMENT
YOUR JUSTLIST MUST FOLLOW THE JSON SPECIFICATION
https://github.com/justswaporg/justlists/blob/main/example.justlists.ts
Checklist

[x] I understand that this is not the place to request a token listing.
[x] I have tested that my JustList is compatible by pasting the URL into the add a list UI at justswap.org.
[x] I understand that filing an issue or adding liquidity does not guarantee addition to the justlists website.

Please provide the following information for your token.
JustList URL must be HTTPS.
JustList URL:
JustList Name:
Link to the official homepage of the JustList manager:

0xae273951d1e129b9575a2d881172d1b7ec4a795f

Please submit the correct information, invalid information will cause the issue to close


Webserver code not handling TP-Link loop detection

(UPDATE)
I turned off the "loop detect" option on the hub and the STM board connected and I was able to connect to the webserver.
I am not sure why this packets were seemingly disabling the process but was unable to connect and obtain an IP address until I disabled the loop detect.

I recently built the step-7 web server project and connected it to my home network.  My hard wired network is running through a TP-Link vlan switch.  I am running on a Nucleo F429-ZI board.  The log messages below appear to indicate that the parser is not handling VLAN tagged frames (last line below) properly.  This is not an area of expertise for me and I am wondering if there is a setting I need to tweak or some change in the code that i can make to get around this issue?  I am new to this so please forgive me if this is not the proper method to ask a question.

A fix is coming soon.

Error not caused by loop detect packets. I am not sure why it began working but all us okay now.


->setRequired() on selectize with liveformvalidation.js dont show the message

Hello,
i am using seletize with  liveformvalidation.js for nette and when i set ->setRequired() on seletize message is not displayed (form does not send - that correct).
https://github.com/contributte/live-form-validation
Thank you

More specification, when is used select mode, everything is fine, but when i use full mode error message is not displayed neither in html there is only empty span. So its not even generate html of that error.


plagiarism at its finest.

Stolen script. This script is stolen from bypassi and doesn't work anymore. nice plagiarism losers.

Did you even read the thing that says read me? I don't take credit.
#LearnToRead

i cant not upgrade on a school administered chromebook for 1, 2 if the script is patched via a version, say so, 3 aint no way u using #learntoread

you wrong.
for 1, your school issues out software updates. If your V120 then it is patched, likely.

is it possible to revert to an earlier version? My Chromebook updates on its own when I turn it off, so I couldn't stop it from doing this.

The insecurly exploit does seem to work for a while after the tab is opened, but it doesn't last.

so chromebook version 120?

not sure
where do I see what version it is?

It works with 120, idk what his issue is. I updated mine to 120 accidently.

It doesn't work for me after the new update, and we have to update because it gets really laggy without it.

https://github.com/HXCHS/HackersX-Unblocked-Websites?tab=readme-ov-file#additional-sites

these are really laggy and not good to use


Custom navigator logo is not displayed

The navigator logo will not be displayed, if the DTM is used with a custom branding. (v. 1014.0.7)

Logo in left navigation menu is meant.
On dev environment, the logo from defined in a custom branding can be seen. Needs to be checked if this has already been fixed:

Working as expected as specified by Christian. Hence closing this issue.


fixed error message outside namespace

Codecov Report

Merging #211 (e41d89f) into main (c1a55c1) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##               main     #211   +/-   ##
=========================================
  Coverage     92.32%   92.32%
  Complexity      293      293
=========================================
  Files            51       51
  Lines           756      756
=========================================
  Hits            698      698
  Misses           58       58

Impacted Files
Coverage Δ

...n/ForClasses/NotHaveDependencyOutsideNamespace.php
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c1a55c1...e41d89f. Read the comment docs.


Ollama: only sometimes running on GPU

Describe the bug
I have installed ollama with the option services.ollama.acceleration = "cuda";. This caused the package to be built on my system, as opposed to being downloaded from a binary cache. After this, it ran very fast, as expected.
After the next reboot, it regressed to running slowly, utilizing the CPU.
After the next reboot, it ran fast again.
After putting the PC to sleep for lunch break, it runs slowly on CPU again.
Steps To Reproduce
Steps to reproduce the behavior:
It's flaky. Sometimes it uses the GPU, sometimes not
Expected behavior
It should always run on the GPU
Screenshots
N/A
Additional context
NVIDIA RTX 4070, configured for offload mode
Notify maintainers
@abysssol @onny @marcusramberg
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.32, NixOS, 24.05 (Uakari), 24.05.675.805a384895c6`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.2`
 - channels(root): `"home-manager-24.05.tar.gz, nixos-24.05"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Add a :+1: reaction to issues you find important.

Whats your Ollama version you're currently using?

Whats your Ollama version you're currently using?

The one packaged in nixos-24.05, that's 0.1.38

The nature of this problem makes me think it could be a bug in upstream ollama; I'm not sure if this has anything to do with nix or not.  I think it could be a good idea to create an issue on ollama's repo.
If you want, you could try building the unstable version of ollama, which is currently 0.1.42. Maybe that won't have this issue? If so, it might be time for me to backport the more recent version of ollama to stable nixos.

The nature of this problem makes me think it could be a bug in upstream ollama; I'm not sure if this has anything to do with nix or not. I think it could be a good idea to create an issue on ollama's repo.
If you want, you could try building the unstable version of ollama, which is currently 0.1.42. Maybe that won't have this issue? If so, it might be time for me to backport the more recent version of ollama to stable nixos.

I'll give it a try

I've installed 0.1.42 and it has the same issue, plus it often starts to produce garbled output or even crashes, requiring to restart the ollama service. So I'm back to 0.1.38.
Some research yielded the command sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm from here as a possible fix for GPU discovery. On first sight, it seems to work

The link seems to indicate that this is actually due to an nvidia driver bug, so this is a little outside of what I can meaningfully fix.
However, I could add something like this to ollama's service module as a workaround:
 powerManagement.powerUpCommands = lib.mkIf (cfg.acceleration == "cuda")
    "rmmod nvidia_uvm && modprobe nvidia_uvm";

I would appreciate it if you tested it first to confirm that it actually solves your problem, before I add it to nixpkgs.
Just add the following to your configuration.nix or equivalent file and rebuild nixos:
 powerManagement.powerUpCommands = "rmmod nvidia_uvm && modprobe nvidia_uvm";

I'm not sure this is the right place to run this command, as ollama "moves" from GPU to CPU at random times, not per session

Really? In this issue description you mention the behavior changing after rebooting the machine and after putting it to sleep, which seems to agree with what is mentioned by ollama's documentation.

On linux, after a suspend/resume cycle, sometimes Ollama will fail to discover your NVIDIA GPU, and fallback to running on the CPU.

According to it's documentation, powerManagement.powerUpCommands also runs "when it resumes from suspend or hibernation". It seems to be the best place I can find to reload the kernel module.

Yes, agreed, if that's how powerManagement.powerUpCommands works that would seem to be the right place. Unfortunately, ollama just moved itself to CPU just after a longer pause, without reboot or suspend. It seems to just happen randomly all the time

Ah, that's unfortunate. Then it seems you'll simply have to run that command whenever cuda stops working. Ollama's recent release 0.1.44 mentions something seemingly relevant, though.

Fixed certain cases where Nvidia GPUs would not be detected and reported as compute capability 1.0 devices

I wonder if that might fix your problem? I opened pr #319783, I expect it'll be available in nixpkgs-unstable within the next week. It might be worth testing 0.1.44 to see if it helps. In case you don't know, you can use ollama's nixos service with the unstable package.
services.ollama = {
  enable = true;
  acceleration = "cuda";
  # `unstable` will have to be created from the `nixpkgs-unstable` channel or flake input
  package = unstable.ollama;
};

If you do end up testing 0.1.44 and it doesn't help, you probably should open an issue on ollama's repo, unless there's already a relevant tracking issue.
But, since this seems to be a known bug in ollama (or the nvidia drivers), I feel that this issue should be closed, as there's not much I can do to help; I just maintain the nixos package of ollama, not ollama itself. Feel free to continue commenting, I'll help if I can, but I feel like I can't do anything for you to fix this problem.


Variable tx.ua_hash and IP collection set but not used in CRS core

Hi there!
tx.ua_hash, initialized in 901318, is just collected in 901321 inside the IP collection. The latter is then never used (I can't find any IP: occurrence since the removal of rules related to IP reputation and DOS from the core).
Could it be reasonable to clean up both the variable and the initcol from the core and maybe add them to just some related plugins? It would avoid SHA1 computation (and storage) of the User-Agent for every request, regardless of whether it is used or not.

Basically, I'm raising again https://github.com/coreruleset/coreruleset/issues/377 (back in the day I think that it was still used)
Actually, not a bug, just a clean-up / little performance improvement.
CRS version (e.g., v3.2.0): v4.0/dev.

Initial conversation with @airween: https://github.com/coreruleset/coreruleset/pull/3043#issuecomment-1363884977.
Many thanks! Happy to provide a PR if seems okay to clean them up

A quick idea: we should add a condition to these rules (901318 and 901321), which checks a plus TX variable (eg. tx.init_ip_coll), which could be set up in crs-setup.conf.

I agree this is in need of a cleanup.
There are pros and cons to @airweens proposal.
Con: CRS does not do sessions anymore, let's not initiate them at all.
Pro: Plugins may want to use sessions and if they do it's better CRS initializes them.
I'm leaning slightly on the CRS-should-init side. But with a config item like @airween proposes.

Are we discussing this more broadly? Is this blocking for v4?

Depends on the solution we chose. :)
Solution 1: It's a plugin problem, plugins need to deal with this
We rip out 901318 and 901321 and we're done.
Solution 2: We make the initialization configurable, off by default, so plugin do not need to coordinate.
V4 depends on this minor change then.
Let's take a decision in the chat tonight.


Issue when running npx ramadan-cli

I'm getting Cannot find module 'cli-table3' error when running npx ramadan-cli
Node version:  13.7.0
npx version: 6.14.4
npm version: 6.14.4
(also ramadan mubarak 🙏 )

log-symbols

Already, fixed. Thanks for the report. Apparently I'm asleep lol.


Training code release?

Thank you for your amazing job~
One question, would you release your training code?

@xjcvip007 It is on the schedule. But before that, it should be granted by the lab leader. Thanks for your interest in our work.

Hi, thank you for your job, do you have plans for sharing training code?

Hi, thank you for your job, do you have plans for sharing training code?

Hi, thank you for your job, do you have plans for sharing training code?

@cleardusk is it possible for you to give pseudo code for dataset preparation for 199 shape and 29 expression Dimensions?


Codecov integration

added new codecov.yml to upload the Icov.info file results into codecov.
added a new script to generate the  lcov.info file and trigger a repository dispatch.
updated cirrus.yml file to generate the lcov file when running tests for the master branch.

Will commit ignoring the failed jobs, since it is not error in the ci files.